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Preface 


Overview 


The objective of this book is to give the reader a flavour of discrete mathematics 
and its applications to the computing field. The goal is to provide a broad and 
accessible guide to the fundamentals of discrete mathematics, and to show how it 
may be applied to various areas in computing such as cryptography, coding theory, 
formal methods, language theory, computability, artificial intelligence, the theory of 
databases and software reliability. The emphasis is on both theory and applications, 
rather than on the study of mathematics for its own sake. 

There are many existing books on discrete mathematics, and while many of these 
provide more in-depth coverage on selected topics, this book is different in that it 
aims to provide a broad and accessible guide to the reader, and to show the rich 
applications of discrete mathematics in a wide number of areas in the computing 
field. 

Each chapter of this book could potentially be a book in its own right, and so 
there are limits to the depth of coverage. However, the author hopes that this book 
will motivate and stimulate the reader, and encourage further study of the more 
advanced texts. 


Organization and Features 


Chapter | discusses the contributions made by early civilizations to computing. 
This includes work done by the Babylonians, Egyptians and Greeks. The Egyptians 
applied mathematics to solving practical problems such as the construction of 
pyramids. The Greeks made major contributions to mathematics and geometry. 

Chapter 2 provides an introduction to fundamental building blocks in discrete 
mathematics including sets, relations and functions. A set is a collection of 
well-defined objects and it may be finite or infinite. A relation between two sets A 
and B indicates a relationship between members of the two sets, and is a subset 
of the Cartesian product of the two sets. A function is a special type of relation such 
that for each element in A, there is at most one element in the codomain B. 
Functions may be partial or total and injective, surjective or bijective. 
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Chapter 3 presents the fundamentals of number theory, and discusses prime 
number theory and the greatest common divisor and least common multiple of two 
numbers. We also discuss the representation of numbers on a computer. 

Chapter 4 discusses mathematical induction and recursion. Induction is a com- 
mon proof technique in mathematics, and there are two parts to a proof by induction 
(the base case and the inductive step). We discuss strong and weak induction, and 
we discuss how recursion is used to define sets, sequences and functions. This leads 
us to structural induction, which is used to prove properties of recursively defined 
structures. 

Chapter 5 discusses sequences and series and permutations and combinations. 
Arithmetic and geometric sequences and series are discussed, and we discuss 
applications of geometric sequences and series to the calculation of compound 
interest and annuities. 

Chapter 6 discusses algebra and we discuss simple and simultaneous equations, 
including the method of elimination and the method of substitution to solve 
simultaneous equations. We show how quadratic equations may be solved by 
factorization, completing the square or using the quadratic formula. We present the 
laws of logarithms and indices. We discuss various structures in abstract algebra, 
including monoids, groups, rings, integral domains, fields and vector spaces. 

Chapter 7 discusses automata theory, including finite state machines, pushdown 
automata and Turing machines. Finite-state machines are abstract machines that are 
in only one state at a time, and the input symbol causes a transition from the current 
state to the next state. Pushdown automata have greater computational power than 
finite-state machines, and they contain extra memory in the form of a stack from 
which symbols may be pushed or popped. The Turing machine is the most powerful 
model for computation, and this theoretical machine is equivalent to an actual 
computer in the sense that it can compute exactly the same set of functions. 

Chapter 8 discusses matrices including 2 x 2 and general m x n matrices. 
Various operations such as the addition and multiplication of matrices are con- 
sidered, and the determinant and inverse of a matrix is discussed. The application of 
matrices to solving a set of linear equations using Gaussian elimination is 
considered. 

Chapter 9 discusses graph theory where a graph G = (V, E) consists of vertices 
and edges. It is a practical branch of mathematics that deals with the arrangements 
of vertices and edges between them, and it has been applied to practical problems 
such as the modelling of computer networks, determining the shortest driving route 
between two cities and the travelling salesman problem. 

Chapter 10 discusses cryptography, which is an important application of number 
theory. The code-breaking work done at Bletchley Park in England during the 
Second World War is discussed, and the fundamentals of cryptography, including 
private and public key cryptosystems, are discussed. 
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Chapter 11 presents coding theory and is concerned with error detection and 
error correction codes. The underlying mathematics of coding theory is abstract 
algebra, including group theory, ring theory, fields and vector spaces. 

Chapter 12 discusses language theory and we discuss grammars, parse trees and 
derivations from a grammar. The important area of programming language 
semantics is discussed, including axiomatic, denotational and operational 
semantics. 

Chapter 13 discusses computability and decidability. The Church—Turing thesis 
states that anything that is computable is computable by a Turing machine. Church 
and Turing showed that mathematics is not decidable, in that there is no mechanical 
procedure (i.e. algorithm) to determine whether an arbitrary mathematical propo- 
sition is true or false, and so the only way is to determine the truth or falsity of a 
statement is try to solve the problem. 

Chapter 14 presents a short history of logic, and we discuss Greek contributions 
to syllogistic logic, stoic logic, fallacies and paradoxes. Boole’s symbolic logic and 
its application to digital computing is discussed, and we consider Frege’s work on 
predicate logic. 

Chapter 15 provides an introduction to propositional and predicate logic. 
Propositional logic may be used to encode simple arguments that are expressed in 
natural language, and to determine their validity. The nature of mathematical proof 
is discussed, and we present proof by truth tables, semantic tableaux and natural 
deduction. Predicate logic allows complex facts about the world to be represented, 
and new facts may be determined via deductive reasoning. Predicate calculus 
includes predicates, variables and quantifiers, and a predicate is a characteristic or 
property that the subject of a statement can have. 

Chapter 16 presents some advanced topics in logic including fuzzy logic, tem- 
poral logic, intuitionistic logic, undefined values, theorem provers and the appli- 
cations of logic to AI. Fuzzy logic is an extension of classical logic that acts as a 
mathematical model for vagueness. Temporal logic is concerned with the expres- 
sion of properties that have time dependencies, such as properties about the past, 
present and future. Intuitionism was a controversial theory on the foundations of 
mathematics based on a rejection of the law of the excluded middle, and an 
insistence on constructive existence. We discuss three approaches to deal with 
undefined values, including the logic of partial functions; Dijkstra’s approach with 
his cand and cor operators; and Parnas’ approach which preserves a classical 
two-valued logic. 

Chapter 17 discusses the nature of proof and theorem proving, and we discuss 
automated and interactive theorem provers. We discuss the nature of formal 
mathematical proof, and consider early attempts at the automation of proof in the 
1960s including the Logic Theorist (LT) and the Geometry Machine. 
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Chapter 18 provides an introduction to the important field of software engi- 
neering. The birth of the discipline was at the Garmisch conference in Germany in 
the late 1960s. The extent to which mathematics should be employed in software 
engineering is discussed, and this remains a topic of active debate. We discuss some 
of the early mathematical contributions to software engineering including the work 
of Floyd and Hoare. 

Chapter 19 discusses software reliability and dependability, and covers topics 
such as software reliability, the Cleanroom methodology, system availability, safety 
and security critical systems and dependability engineering. Software reliability is 
the probability that the program works correctly without failure for a period of time, 
and is generally expressed as the mean time to failure. 

Chapter 20 discusses formal methods, which consist of a set of techniques that 
provide an extra level of confidence in the correctness of the software. They may be 
employed to formally state the requirements of the proposed system, and to derive a 
program from its mathematical specification. They may be used to give a rigorous 
proof that the implemented program satisfies its specification. 

Chapter 21 presents the Z specification language, which is one of the most 
widely used formal methods. It was developed at Oxford University in the U.K. 

Chapter 22 discusses statistics which is an empirical science that is concerned 
with the collection, organization, analysis, interpretation and presentation of data. 
We discuss sampling; the average and spread of a sample; the abuse of statistics; 
frequency distributions; variance and standard deviation; correlation and regression; 
statistical inference; and hypothesis testing. 

Chapter 23 discusses probability which is a branch of mathematics that is 
concerned with measuring uncertainty and random events. We discuss discrete and 
continuous random variables; probability distributions such as the binomial and 
normal distributions; variance and standard deviation; confidence intervals; tests of 
significance; the central limit theorem; Bayesianism; and queueing theory. 

Chapter 24 discusses operations research which is a multi-disciplinary field that 
is concerned with the application of mathematical and analytic techniques to assist 
in decision-making. It employs techniques such as mathematical modelling, sta- 
tistical analysis and mathematical optimization as part of its goal to achieve optimal 
(or near-optimal) solutions to complex decision-making problems. 

Chapter 25 discusses basic financial mathematics, and we discuss simple and 
compound interest, annuities and mortgages. We discuss the basic mathematics 
used in calculating simple and compound interest, as well as calculating the present 
or future value of a payment. We discuss the mathematics of annuities (a sequence 
of fixed equal payments made over a period of time), and this is the usual way in 
which a loan or mortgage is paid back. 


Preface xi 
Audience 


The audience of this book includes computer science students who wish to gain a 
broad and accessible overview of discrete mathematics and its applications to the 
computing field. The book will also be of interest to students of mathematics who 
are curious as to how discrete mathematics is applied to the computing field. The 
book will also be of interest to the motivated general reader. 
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Mathematics in Civilization 


1.1. Introduction 


It is difficult to think of Western society today without modern technology. The last 
decades of the twentieth century have witnessed a proliferation of high-tech com- 
puters, mobile phones, text messaging, the Internet and the World Wide Web. 
Software is now pervasive, and it is an integral part of automobiles, aeroplanes, 
televisions and mobile communication. The pace of change as a result of all this 
new technology has been extraordinary. Today, consumers may book flights over 
the World Wide Web as well as keep in contact with family members in any part of 
the world via e-mail or mobile phone. In previous generations, communication 
often involved writing letters that took months to reach the recipient. 

Communication improved with the telegrams and the telephone in the late 
nineteenth century. Communication today is instantaneous with text messaging, 
mobile phones and e-mail, and the new generation probably views the world of 
their parents and grandparents as being old-fashioned. 

The new technologies have led to major benefits’ to society and to improvements 
in the standard of living for many citizens in the Western world. It has also reduced 
the necessity for humans to perform some of the more tedious or dangerous manual 
tasks, as computers may now automate many of these. The increase in productivity 
due to the more advanced computerized technologies has allowed humans, at least in 
theory, the freedom to engage in more creative and rewarding tasks. 


‘Of course, it is essential that the population of the world moves towards more sustainable 
development to ensure the long-term survival of the planet for future generations. This involves 
finding technological and other solutions to reduce greenhouse gas emissions as well as moving to 
a carbon neutral way of life. The solution to the environmental issues will be a major challenge for 
the twenty-first century. 
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Early societies had a limited vocabulary for counting: e.g. ‘one, two, three, 
many’ is associated with some primitive societies, and indicates primitive com- 
putation and scientific ability. It suggests that there was no need for more sophis- 
ticated arithmetic in the primitive culture as the problems dealt with were 
elementary. These early societies would typically have employed their fingers for 
counting, and as humans have 5 fingers on each hand and five toes on each foot, the 
obvious bases would have been 5, 10 and 20. Traces of the earlier use of the base 20 
system are still apparent in modern languages such as English and French. This 
includes phrases such as ‘three score’ in English and ‘quatre vingt’ in French. 

The decimal system (base 10) is used today in Western society, but the base 60 
was common in computation circa 1500 B.C. One example of the use of base 60 
today is the subdivision of hours into 60 min, and the subdivision of minutes into 
60s. The base 60 system (i.e. the sexagesimal system) is inherited from the 
Babylonians [1]. The Babylonians were able to represent arbitrarily large numbers 
or fractions with just two symbols. The binary (base 2) and hexadecimal (base 16) 
systems play a key role in computing (as the machine instructions that computers 
understand are in binary code). 

The achievements of some of these ancient societies were spectacular. The 
archaeological remains of ancient Egypt such as the pyramids at Giza and the 
temples of Karnak and Abu Simbel are impressive. These monuments provide an 
indication of the engineering sophistication of the ancient Egyptian civilization, 
despite the fact that Egyptian mathematics was cumbersome. The objects found in 
the tomb of Tutankhamun? are now displayed in the Egyptian museum in Cairo, 
and demonstrate the artistic skill of the Egyptians. 

The Greeks made major contributions to Western civilization including contri- 
butions to Mathematics, Philosophy, Logic, Drama, Architecture, Biology and 
Democracy.* The Greek philosophers considered fundamental questions such as 
ethics, the nature of being, how to live a good life, and the nature of justice and 
politics. The Greek philosophers include Parmenides, Heraclitus, Socrates, Plato 
and Aristotle. The Greeks invented democracy, and their democracy was radically 


Tutankhamun was a minor Egyptian pharaoh who reigned after the controversial rule of 
Akhenaten. Tutankhamun’s tomb was discovered by Howard Carter in the Valley of the Kings, 
and the tomb was intact. The quality of the workmanship of the artefacts found in the tomb is 
extraordinary, and a visit to the Egyptian museum in Cairo is memorable. 

3The origin of the word ‘democracy’ is from demos (Sn10¢) meaning people and kratos (Kpatos) 
meaning rule. That is, it means rule by the people and it was introduced into Athens following the 
reforms introduced by Cleisthenes. He divided the Athenian city-state into 30 areas, where 20 of 
these areas were inland or along the coast and ten were in Attica itself. Fishermen lived mainly in 
the ten coastal areas; farmers in the ten inland areas; and various tradesmen in Attica. Cleisthenes 
introduced ten new clans where the members of each clan came from one coastal area, one inland 
area on one area in Attica. He then introduced a Boule (or assembly) which consisted of 500 
members (50 from each clan). Each clan ruled for "ho th of the year. 
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different from today’s representative democracy.* The sophistication of Greek 
architecture and sculpture is evident from the Parthenon on the Acropolis, and the 
Elgin marbles> that are housed today in the British Museum, London. 

The Hellenistic’ period commenced with Alexander the Great and led to the 
spread of Greek culture throughout most of the known world. The city of 
Alexandria became a centre of learning and knowledge during the Hellenistic 
period. Its scholars included Euclid who provided a systematic foundation for 
geometry. His work is known as “The Elements”, and it consists of 13 books. The 
early books are concerned with the construction of geometric figures, number 
theory and solid geometry. 

There are many words of Greek origin that are part of the English language. 
These include words such as psychology that is derived from two Greek words: 
psyche (yo€s) and logos (Aoyoc). The Greek word ‘psyche’ means mind or soul, 
and the word ‘logos’ means an account or discourse. Other examples are anthro- 
pology derived from ‘anthropos’ (avtpopoc) and ‘logos’ (Aoyoc). 

The Romans were influenced by the Greek culture. The Romans built aqueducts, 
viaducts and amphitheatres. They also developed the Julian calendar, formulated 
laws (Jex) and maintained peace throughout the Roman Empire (pax Romano). The 
ruins of Pompeii and Herculaneum demonstrate their engineering capability. Their 
numbering system is still employed in clocks and for page numbering in docu- 
ments. However, it is cumbersome for serious computation. The collapse of the 
Roman Empire in Western Europe led to a decline in knowledge and learning in 
Europe. However, the eastern part of the Roman Empire continued at Con- 
stantinople until it was sacked by the Ottomans in 1453. 


‘The Athenian democracy involved the full participations of the citizens (ie. the male adult 
members of the city-state who were not slaves), whereas in representative democracy the citizens 
elect representatives to rule and represent their interests. The Athenian democracy was chaotic and 
could be easily influenced by individuals who were skilled in rhetoric. There were teachers (known 
as the Sophists) who taught wealthy citizens rhetoric in return for a fee. The origin of the word 
‘sophist’ is the Greek word co@oc¢ meaning wisdom, and one of the most well known of the 
sophists was Protagoras. The problems with the Athenian democracy led philosophers such as 
Plato to consider alternate solutions such as rule by philosopher kings. This totalitarian utopian 
state is described in Plato’s Republic. 

>The Elgin marbles are named after Lord Elgin who was the British ambassador to Greece (which 
was then part of the Ottoman Empire), and he removed them (at his own expense) over several 
years from the Parthenon in Athens to London during the first decade of the nineteenth century. 
The marbles show the Pan-Athenaic festival that was held in Athens in honour of the goddess 
Athena after whom Athens is named. 

°The origin of the word Hellenistic is from Hellene (‘EAAnv) meaning Greek. 
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1.2 The Babylonians 


The Babylonian’ civilization flourished in Mesopotamia (in modern Iraq) from 
about 2000 B.C. until about 300 B.C. Various clay cuneiform tablets containing 
mathematical texts were discovered and later deciphered in the nineteenth century 
[2]. These included tables for multiplication, division, squares, cubes and square 
roots, and the measurement of area and length. Their calculations allowed the 
solution of a linear equation and one root of a quadratic equation to be determined. 
The late Babylonian period (c. 500 B.C.) includes work on astronomy. 

They recorded their mathematics on soft clay using a wedge-shaped instrument 
to form impressions of the cuneiform numbers.* The clay tablets were then baked in 
an oven or by the heat of the sun. They employed just two symbols (1 and 10) to 
represent numbers, and these symbols were then combined to form all other 
numbers. They employed a positional number system? and used the base 60 system. 
The symbol representing | could also (depending on the context) represent 60, 60°, 
60°, etc. It could also mean Vets yen and so on. There was no zero employed in 
the system and there was no decimal point (no ‘sexagesimal point’), and therefore 
the context was essential. 


7< Y 


The example above illustrates the cuneiform notation and represents the number 
60 + 10 + 1 = 71. The Babylonians used the base 60 system, and this base is still 
in use today in the division of hours into minutes and the division of minutes into 
seconds. One possible explanation for the use of the base 60 notation is the ease of 
dividing 60 into parts. It is divisible by 2, 3, 4, 5, 6, 10, 12, 15, 20 and 30. They 
were able to represent large and small numbers and had no difficulty in working 
with fractions (in base 60) and in multiplying fractions. The Babylonians main- 
tained tables of reciprocals (i.e. nes n= 1, ... 59) apart from numbers like 7, 11, etc. 
which cannot be written as a finite sexagesimal expansion (i.e. 7 and 11 are not of 
the form 2%3°5"), 

The modern sexagesimal notation [1] 1; 24, 51, 10 represents the number 
1+ 24/60 + 51/3600 + 10/216,000 = 1+0.4 + 0.0141666 + 0.0000462 = 1.4142129. 
This is the Babylonian representation of the square root of 2. They performed 
multiplication as follows: e.g. consider 20 * sqrt(2) = (20) * (1; 24,51, 10): 


The hanging gardens of Babylon were one of the seven wonders of the ancient world. 

Henry Rawlinson played an important role in the decipherment of cuneiforms, and especially for 
making a copy of the large Behistun inscription (the equivalent of the Rosetta stone for 
Assyriologists) that recorded the victory of the Persian king Darius I over those who rebelled 
against him in three languages. Edward Hicks and others played a key role in deciphering the 
inscription. 

°A positional numbering system is a number system where each position is related to the next by a 
constant multiplier. The decimal system is an example: e.g. 546 = 5* 107 + 4* 10' + 6. 
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Fig. 1.1 The Plimpton 322 Tablet 


20 «1 = 20 

20%; 24 = 20 « 24/60 = 8 

20 * 51/3600 = 51/180 = 17/60 =; 17 

20 * 10/216,000 = 3/3600 + 20/216,000 =; 0, 3, 20. 


Hence, the product 20 « sqrt(2) = 20; + 8; +;17+;0,3, 20 = 28; 17,3, 20. 

The Babylonians appear to have been aware of Pythagoras’ theorem about 
1000 years before the time of Pythagoras. The Plimpton 322 tablet (Fig. 1.1) 
records various Pythagorean triples, i.e. triples of numbers (a, b, c) where 
a’ +b* =’. It dates from approximately 1700 B.C. 

They developed an algebra to assist with problem-solving, and their algebra 
allowed problems involving length, breadth and area to be discussed and solved. 
They did not employ notation for the representation of unknown values (e.g. let x be 
the length and y be the breadth), and instead they used words like ‘length’ and 
‘breadth’. They were familiar with and used square roots in their calculations, and 
they were familiar with techniques that allowed one root of a quadratic equation to 
be solved. 

They were familiar with various mathematical identities such as (a+b)= 
(a*+2ab+b?) as illustrated geometrically in Fig. 1.2. They also worked on 
astronomical problems, and they had mathematical theories of the cosmos to make 
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Fig. 1.2 Geometric 
representation of 


(a+b)= (a? + 2ab +b?) 


predictions of when eclipses and other astronomical events would occur. They were 
also interested in astrology, and they associated various deities with the heavenly 
bodies such as the planets, as well as the sun and moon. They associated various 
clusters of stars with familiar creatures such as lions, goats and so on. 

The Babylonians used counting boards to assist with counting and simple cal- 
culations. A counting board is an early version of the abacus, and it was usually 
made of wood or stone. The counting board contained grooves that allowed beads 
or stones that could be moved along the groove. The abacus differs from counting 
boards in that the beads in abaci contain holes that enable them to be placed in a 
particular rod of the abacus. 


1.3. The Egyptians 


The Egyptian civilization developed along the Nile from about 4000 B.C., and the 
pyramids were built around 3000 B.C. They used mathematics to solve practical 
problems such as measuring time; measuring the annual Nile flooding; calculating 
the area of land; book keeping and accounting; and calculating taxes. They 
developed a calendar circa 4000 B.C., which consisted of 12 months with each 
month having 30 days. There were then five extra feast days to give 365 days in a 
year. Egyptian writing commenced around 3000 B.C. and is recorded on the walls 
of temples and tombs.'? A reed-like parchment termed ‘papyrus’ was used for 
writing, and three Egyptian writing scripts were employed. These were hiero- 
glyphics, the hieratic script and the demotic script. 

Hieroglyphs are little pictures and are used to represent words, alphabetic 
characters as well as syllables or sounds. Champollion deciphered hieroglyphics 
with his work on the Rosetta stone. This object was discovered during the Napo- 
leonic campaign in Egypt, and it is now in the British Museum in London. It 
contains three scripts: Hieroglyphics, Demotic script and Greek. The key to its 
decipherment was that the Rosetta stone contained just one name “Ptolemy” in the 


‘The decorations of the tombs in the Valley of the Kings record the life of the pharaoh including 
his exploits and successes in battle. 
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Fig. 1.3 Egyptian numerals 


Greek text, and this was identified with the hieroglyphic characters in the car- 
touche’! of the hieroglyphics. There was just one cartouche on the Rosetta stone, 
and Champollion inferred that the cartouche represented the name “Ptolemy”. He 
was familiar with another multilingual object that contained two names in the 
cartouche. One he recognized as Ptolemy and the other he deduced from the Greek 
text as “Cleopatra”. This led to the breakthrough in the translation of hieroglyphics 
[1]. 

The Rhind Papyrus is a famous Egyptian papyrus on mathematics. The Scottish 
Egyptologist, Henry Rhind, purchased it in 1858, and it is a copy created by an 
Egyptian scribe called Ahmose’* around 1832 B.C. It contains examples of many 
kinds of arithmetic and geometric problems, and students may have used it as a 
textbook to develop their mathematical knowledge. This would allow them to 
participate in the pharaoh’s building programme. 

The Egyptians were familiar with geometry, arithmetic and elementary algebra. 
They had techniques to find solutions to problems with one or two unknowns. 
A base 10 number system was employed with separate symbols for one, ten, a 
hundred, a thousand, a ten thousand, a hundred thousand and so on. These hiero- 
glyphic symbols are represented in Fig. 1.3. 

For example, the representation of the number 276 in Egyptian hieroglyphics is 
described in Fig. 1.4. 

The addition of two numerals is straightforward and involves adding the indi- 
vidual symbols, and where there are ten copies of a symbol it is then replaced by a 
single symbol of the next higher value. The Egyptians employed unit fractions (e.g. 
1/n where n is an integer). These were represented in hieroglyphs by placing the 
symbol representing a ‘mouth’ above the number. The symbol mouth is used to 
represent part of (or a fraction of) a number. For example, the representation of the 
number Wage is described in Fig. 1.5. 

The problems on the papyrus included the determination of the angle of the slope 
of the pyramid’s face. They were familiar with trigonometry including sine, cosine, 
tangent and cotangent, and they knew how to build right angles into their structures 


"'The cartouche surrounded a group of hieroglyphic symbols enclosed by an oval shape. 
Champollion’s insight was that the group of hieroglyphic symbols represented the name of the 
Ptolemaic pharaoh “Ptolemy”. 

The Rhind papyrus is sometimes referred to as the Ahmes papyrus in honour of the scribe who 
wrote it in 1832 B.C. 
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Fig. 1.4 Egyptian representation of a number 
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Fig. 1.5 Egyptian representation of a fraction 


by using the ratio 3:4:5. The Rhind papyrus also considered problems such as the 
calculation of the number of bricks required for part of a building project. Multi- 
plication and division were cumbersome in Egyptian mathematics as they could 
only multiply and divide by two, and this limited the mathematical progress made 
by the Egyptians. 

Suppose they wished to multiply a number n by 7. Then n * 7 is determined by 
nx2+n*2+n*2-+n. Similarly, if they wished to divide 27 by 7 they would 
note that 7 * 2 + 7 = 21 and that 27 — 21 = 6 and therefore the answer was 3 eh. 
Egyptian mathematics was cumbersome and the writing of their mathematics was 
long and repetitive. For example, they wrote a number such as 22 by 
10+ 104+1+1. 

The Egyptians calculated the approximate area of a circle by calculating the area 
of a square °/y of the diameter of a circle. That is, instead of calculating the area in 
terms of our familiar r*, their approximate calculation yielded 
(8/9 « 2r)’= 256/81r? or 3.1672. Their approximation of m was 7°°%/g, or 3.16. 
They were able to calculate the area of a triangle and volumes. The Moscow 
papyrus includes a problem to calculate the volume of the frustum. 
The formula for the volume of a frustum of a square pyramid’? was given by 
V=1/3 (by + bib + b3) and when b> is O, then the well-known formula for the 
volume of a pyramid is given as '/; hb. 


'SThe length of a side of the bottom base of the pyramid is b,, and the length of a side of the top 
base is bo. 
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1.4 The Greeks 


The Greeks made major contributions to Western civilization including mathe- 
matics, logic, astronomy, philosophy, politics, drama and architecture. The Greek 
world of 500 B.C. consisted of several independent city-states such as Athens and 
Sparta, and various city-states in Asia Minor. The Greek polis (moAt0) or city-state 
tended to be quite small, and consisted of the Greek city and a certain amount of 
territory outside the city-state. Each city-state had political structures for its citizens, 
and some were oligarchs where political power was maintained in the hands of a 
few individuals or aristocratic families. Others were ruled by tyrants (or sole rulers), 
who sometimes took power by force, but who often had a lot of support from the 
public. The tyrants included people such as Solon, Peisistratus and Cleisthenes in 
Athens. 

The reforms by Cleisthenes led to the introduction of the Athenian democracy. 
Power was placed in the hands of the citizens who were male (women or slaves did 
not participate in the Athenian democracy). It was an extremely liberal democracy 
where citizens voted on all important issues. Often, this led to disastrous results as 
speakers who were skilled in rhetoric could exert significant influence. This later led 
to Plato advocating rule by philosopher kings rather than by democracy.'* 

Early Greek mathematics commenced approximately 500-600 B.C. with work 
done by Pythagoras and Thales. Pythagoras was a philosopher and mathematician 
who had spent time in Egypt becoming familiar with Egyptian mathematics. He 
lived on the island of Samos, and formed a secret society known as the 
Pythagoreans. They included men and women and believed in the transmigration of 
souls, and that number was the essence of all things. They discovered the mathe- 
matics for harmony in music with the relationship between musical notes being 
expressed in numerical ratios of small whole numbers. Pythagoras is credited with 
the discovery of Pythagoras’ theorem, although the Babylonians probably knew this 
theorem about 1000 years earlier. The Pythagorean society was dealt with a major 
blow” by the discovery of the incommensurability of the square root of 2: i.e. there 
are no numbers p, g such that \/2 = p/q. 

Thales was a sixth-century (B.C.) philosopher from Miletus in Asia Minor who 
made contributions to philosophy, geometry and astronomy. His contributions to 
philosophy are mainly in the area of metaphysics, and he was concerned with 
questions on the nature of the world. His objective was to give a natural or scientific 
explanation of the cosmos, rather than relying on the traditional supernatural 
explanation of creation in Greek mythology. He believed that there was a single 
substance that was the underlying constituent of the world, and he believed that this 
substance was water. 


'4Plato’s Republic describes his utopian state, and seems to be based on the austere Spartan model. 
The Pythagoreans took a vow of silence with respect to the discovery of incommensurable 
numbers. However, one member of the society is said to have shared the secret result with others 
outside the sect, and an apocryphal account is that he was thrown into a lake for his betrayal and 
drowned. The Pythagoreans obviously took Mathematics seriously back then. 
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He also contributed to mathematics [3], and a well-known theorem in Euclidean 
geometry is named after him. It states that if A, B and C are points on a circle, and 
where the line AC is a diameter of the circle, then the angle <ABC is a right angle. 

The rise of Macedonia led to the Greek city-states being conquered by Philip of 
Macedonia in the fourth century B.C. His son, Alexander the Great, defeated the 
Persian Empire, and extended his empire to include most of the known world. This 
led to the Hellenistic Age with Greek language and culture spreading throughout 
the known world. Alexander founded the city of Alexandra, and it became a major 
centre of learning. However, Alexander’s reign was very short as he died at the 
young age of 33 in 323 B.C. 

Euclid lived in Alexandria during the early Hellenistic period, and he is con- 
sidered the father of geometry and the deductive method in mathematics. His 
systematic treatment of geometry and number theory is published in the 13 books of 
the Elements [4]. It starts from five axioms, five postulates and twenty-three defi- 
nitions to logically derive a comprehensive set of theorems. His method of proof 
was often constructive, in that as well as demonstrating the truth of a theorem the 
proof would often include the construction of the required entity. He also used 
indirect proof to show that there are an infinite number of primes: 


1. Suppose there are a finite number of primes (say n primes). 
2. Multiply all n primes together and add | to form N: 


(N = pj * pr * +++ * Py +1). 


1. N is not divisible by py, po, ..., Py aS dividing by any of these gives a remainder 
of one. 

2. Therefore, N must either be prime or divisible by some other prime that was not 
included in the list. 

3. Therefore, there must be at least n + | primes. 

4. This is a contradiction as it was assumed that there was a finite number of 
primes n. 

5. Therefore, the assumption that there are a finite number of primes is false. 

6. Therefore, there are infinite number of primes. 


Euclidean geometry included the parallel postulate (or Euclid’s fifth postulate). 
This postulate generated interest, as many mathematicians believed that it was 
unnecessary and could be proved as a theorem. It states that: 


Definition 1.1 (Parallel Postulate) If a line segment intersects two straight lines 
forming two interior angles on the same side that sum to less than two right angles, 
then the two lines, if extended indefinitely, meet on that side on which the angles 
sum to less than two right angles. 
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This postulate was later proved to be independent of the other postulates, with 
the development of non-Euclidean geometries in the nineteenth century. These 
include the hyperbolic geometry discovered independently by Bolyai and Loba- 
chevsky, and elliptic geometry developed by Riemann. The standard model of 
Riemannian geometry is the sphere where lines are great circles. 

Euclid’s Elements is a systematic development of geometry starting from the 
small set of axioms, postulates and definitions, leading to theorems logically 
derived from the axioms and postulates. Euclid’s deductive method influenced later 
mathematicians and scientists. There are some jumps in reasoning and the German 
mathematician, David Hilbert, later added extra axioms to address this. 

The Elements contains many well-known mathematical results such as 
Pythagoras’ Theorem, Thales Theorem, Sum of Angles in a Triangle, Prime 
Numbers, Greatest Common Divisor and Least Common Multiple, Euclidean 
Algorithm, Areas and Volumes, Tangents to a Point and Algebra. 

The Euclidean algorithm is one of the oldest known algorithms and is employed 
to produce the greatest common divisor of two numbers. It is presented in the 
Elements but was known well before Euclid. The algorithm to determine the gcd of 
two natural numbers, a and b, is given by 


1. Check if b is zero. If so, then a is the gcd. 
2. Otherwise, the gcd (a, b) is given by gcd (b, a mod BD). 


It is also possible to determine integers p and q such that ap + bg = gcd(a, b). 

The proof of the Euclidean algorithm is as follows. Suppose a and b are two 
positive numbers whose gcd has to be determined, and let r be the remainder when 
a is divided by b. 


Clearly, a = gb+r where q is the quotient of the division. 

Any common divisor of a and b is also a divisor of r (since r = a — qb). 

. Similarly, any common divisor of b and r will also divide a. 

. Therefore, the greatest common divisor of a and b is the same as the greatest 
common divisor of b and r. 

5. The number r is smaller than b, and we will reach r = 0 in finitely many steps. 

6. The process continues until r = 0. 


AWN 


Comment 1.1 


Algorithms are fundamental in computing as they define the procedure by which a 
problem is solved. A computer program implements the algorithm in some pro- 
gramming language. 
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Fig. 1.6 Eratosthenes measurement of the circumference of the earth 


Eratosthenes was a Hellenistic mathematician and scientist who worked at the 
library in Alexandria, which was the largest library in the ancient world. It was built 
cueing the Hellenistic period in the third century B.C. and destroyed by fire in 391 
A.D." 

Eratosthenes devised a system of latitude and longitude, and became the first 
person to estimate the size of the circumference of the earth (Fig. 1.6). His cal- 
culation proceeded as follows: 


1. On the summer solstice at noon in the town of Aswan’” on the Tropic of Cancer 
in Egypt, the sun appears directly overhead. 

2. Eratosthenes believed that the earth was a sphere. 

3. He assumed that rays of light came from the sun in parallel beams and reached 
the earth at the same time. 

4. At the same time in Alexandria, he had measured that the sun would be 7.2° 
south of the zenith. 

5. He assumed that Alexandria was directly north of Aswan. 

6. He concluded that the distance from Alexandria to Aswan was ey er of the 
circumference of the earth. 

7. Distance between Alexandria and Aswan was 5000 stadia (approximately 
800 km). 

8. He established a value of 252,000 stadia or approximately 396,000 km. 


‘©The library in Alexandria is predated by the Royal library of Ashurbanipal which was established 
in Nineveh (the capital of the Assyrian Empire) in the seventh century B.C. The latter contained 
over 30,000 cuneiform tablets including the famous “Epic of Gilgamesh”, and the laws in 
Hammurabi’s code. 

"The town of Aswan is famous today for the Aswan high dam, which was built in the 1960s. 
There was an older Aswan dam built by the British in the late nineteenth century. The new dam led 
to a rise in the water level of Lake Nasser and flooding of archaeological sites along the Nile. 
Several archaeological sites such as Abu Simbel and the temple of Philae were relocated to higher 
ground. 
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Eratosthenes’ calculation was an impressive result for 200 B.C. The errors in his 
calculation were due to the following: 


1. Aswan is not exactly on the Tropic of Cancer but it is actually 55 km north of it. 

2. Alexandria is not exactly north of Aswan, and there is a difference of 3° 
longitude. 

3. The distance between Aswan and Alexandria is 729 km, not 800 km. 

. Angles in antiquity could not be measured with a high degree of precision. 

5. The angular distance is actually 7.08° and not 7.2°. 


Eratosthenes also calculated the approximate distance to the moon and sun and 
he also produced maps of the known world. He developed a very useful algorithm 
for determining all of the prime numbers up to a specified integer. The method is 
known as the Sieve of Eratosthenes and the steps are as follows: 


1. Write a list of the numbers from 2 to the largest number that you wish to test for 
primality. This first list is called A. 

. A second list B is created to list the primes. It is initially empty. 

. The number 2 is the first prime number and is added to the list of primes in B. 

. Strike off (or remove) 2 and all the multiples of 2 from List A. 

. The first remaining number in List A is a prime number, and this prime number 
is added to List B. 

6. Strike off (or remove) this number and all the multiples of this number from 

List A. 
7. Repeat steps 5 through 7 until no more numbers are left in List A. 


nA Wh 


Comment 1.2 


The Sieve of Eratosthenes method is a well-known algorithm for determining prime 
numbers. 

Archimedes was a Hellenistic mathematician, astronomer and philosopher who 
lived in Syracuse in the third century B.C. He discovered the law of buoyancy 
known as Archimedes’ principle: 


The buoyancy force is equal to the weight of the displaced fluid. 


He is believed to have discovered the principle while sitting in his bath. He was 
so overwhelmed with his discovery that he rushed out onto the streets of Syracuse 
shouting “Eureka”, but forgot to put on his clothes to announce the discovery. 

The weight of the displaced liquid will be proportional to the volume of the 
displaced liquid. Therefore, if two objects have the same mass, the one with greater 
volume (or smaller density) has greater buoyancy. An object will float if its buoyant 
force (i.e. the weight of the liquid displaced) exceeds the downward force of gravity 
(i.e. its weight). If the object has exactly the same density as the liquid, then it will 
stay still, neither sinking nor floating upwards. 
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Fig. 1.7. Archimedes in 
thought by Fetti 


For example, a rock is generally a very dense material and will generally not 
displace its own weight. Therefore, a rock will sink to the bottom as the downward 
weight exceeds the buoyancy weight. However, if the weight of the object is less 
than the liquid, it would displace and then it floats at a level where it displaces the 
same weight of liquid as the weight of the object. 

Archimedes (Fig. 1.7) was born in Syracuse’® in the third century B.C. He was a 
leading scientist in the Greco-Roman world, and he is credited with designing 
several innovative machines. 

His inventions include the “Archimedes Screw” which was a screw pump that is 
still used today in pumping liquids and solids. Another of his inventions was the 
“Archimedes Claw”, which was a weapon used to defend the city of Syracuse. It 
was also known as the ‘ship shaker’ and it consisted of a crane arm from which a 
large metal hook was suspended. The claw would swing up and drop down on the 
attacking ship. It would then lift it out of the water and possibly sink it. Another of 
his inventions was said to be the “Archimedes Heat Ray”. This device is said to 
have consisted of a number of mirrors that allowed sunlight to be focused on an 
enemy ship thereby causing it to go on fire. 

He made good contributions to mathematics including developing a good 
approximation to 1, as well as contributions to the positional numbering system, 
geometric series and to maths physics. He also solved several interesting problems: 
e.g. the calculation of the composition of cattle in the herd of the Sun god by 
solving a number of simultaneous Diophantine equations. The herd consisted of 


'8syracuse is located on the island of Sicily in Southern Italy. 


1.4 The Greeks 15 


bulls and cows with one part of the herd consisting of white, second part black, 
third spotted and the fourth brown. Various constraints were then expressed in 
Diophantine equations, and the problem was to determine the precise composition 
of the herd. Diophantine equations are named after Diophantus who worked on 
number theory in the third century. 

There is a well-known anecdote concerning Archimedes and the crown of King 
Hiero II. The king wished to determine whether his new crown was made entirely 
of solid gold, and that the goldsmith had not added substitute silver. Archimedes 
was required to solve the problem without damaging the crown, and as he was 
taking a bath he realized that if the crown was placed in water that the water 
displaced would give him the volume of the crown. From this, he could then 
determine the density of the crown and therefore whether it consisted entirely of 
gold. 

Archimedes also calculated an upper bound of the number of grains of sands in 
the known universe. The largest number in common use at the time was a myriad 
(100 million), where a myriad is 10,000. Archimedes’ numbering system goes up to 
8 * 10'° and he also developed the laws of exponents: i.e. 10710’ = 10°+”. His 
calculation of the upper bound includes the grains of sand not only on each beach 
but also on the earth filled with sand and the known universe filled with sand. His 
final estimate of the upper bound for the number of grains of sand in a filled 
universe was 10%. 

It is possible that he may have developed the odometer,'” and this instrument 
could calculate the total distance travelled on a journey. An odometer is described 
by the Roman engineer Vitruvius around 25 B.C., and it employed a wheel with a 
diameter of 4 feet, and the wheel turned 400 times in every mile.*” The device 
included gears and pebbles and a 400-tooth cogwheel that turned once every mile 
and caused one pebble to drop into a box. The total distance travelled was deter- 
mined by counting the pebbles in the box. 

Aristotle was born in Macedonia and became a student of Plato in Athens 
(Fig. 1.8). Plato had founded a school (known as Plato’s Academy) in Athens in the 
fourth century B.C., and this school remained open until 529 A.D. Aristotle 
founded his own school (known as the Lyceum) in Athens. He was also the tutor of 
Alexander the Great. He made contributions to physics, biology, logic, politics, 
ethics and metaphysics. 


The origin of the word “odometer” is from the Greek words ‘od0C’ (meaning journey) and 
LlETPOV Meaning (measure). 

The figures given here are for the distance of one Roman mile. This is given by 74 * 400 = 
12.56 * 400 = 5024 (which is less than 5280 feet for a standard mile in the Imperial system). 
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Fig. 1.8 Plato and Aristotle 


Aristotle’s starting point to the acquisition of knowledge was the senses, as he 
believed that these were essential to acquire knowledge. This position is the 
opposite of Plato who argued that the senses deceive and should not be relied upon. 
Plato’s writings are mainly in dialogues involving his former mentor Socrates.” 

Aristotle made important contributions to formal reasoning with his develop- 
ment of syllogistic logic. His collected works on logic is called the Organon and it 
was used in his school in Athens. Syllogistic logic (also known as term logic) 
consists of reasoning with two premises and one conclusion. Each premise consists 
of two terms and there is a common middle term. The conclusion links the two 
unrelated terms from the premises. For example: 


*!Socrates was a moral philosopher who deeply influenced Plato. His method of enquiry into 
philosophical problems and ethics was by questioning. Socrates himself maintained that he knew 
nothing (Socratic ignorance). However, from his questioning it became apparent that those who 
thought they were clever were not really that clever after all. His approach obviously would not 
have made him very popular with the citizens of Athens. Chaerephon (a friend of Socrates) had 
consulted the oracle at Delphi to find out who the wisest of all men was, and he was informed that 
there was no one wiser than Socrates. Socrates was sentenced to death for allegedly corrupting the 
youth of Athens, and the sentence was carried out by Socrates being forced to take hemlock (a type 
of poison). The juice of the hemlock plant was prepared for Socrates to drink. 
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Premise 1 All Greeks are Mortal; 
Premise 2 Socrates is a Greek. 
Conclusion Socrates is Mortal. 


The common middle term is ‘Greek’, which appears in the two premises. The 
two unrelated terms from the premises are ‘Socrates’ and ‘Mortal’. The relationship 
between the terms in the first premise is that of the universal: i.e. anything or any 
person that is a Greek is mortal. The relationship between the terms in the second 
premise is that of the particular: i.e. Socrates is a person who is a Greek. The 
conclusion from the two premises is that Socrates is mortal: i.e. a particular rela- 
tionship between the two unrelated terms ‘Socrates’ and ‘Mortal’. 

The syllogism above is a valid syllogistic argument. Aristotle studied the various 
possible syllogistic arguments and determined those that were valid and invalid. 
Syllogistic logic is described in more detail in Chap. 14. Aristotle’s work was 
highly regarded in classical and mediaeval times, and Kant believed that there was 
nothing else to invent in Logic. There was another competing system of logic 
proposed by the Stoics in Hellenistic times: i.e. an early form of propositional logic 
that was developed by Chrysippus~ in the third century B.C. Aristotelian logic is 
mainly of historical interest today. 

Aquinas,” a thirteenth century Christian theologian and philosopher, was deeply 
influenced by Aristotle, and referred to him as the philosopher. Aquinas was an 
empiricist (i.e. he believed that all knowledge was gained by sense experience), and 
he used some of Aristotle’s arguments to offer five proofs of the existence of God. 
These arguments included the Cosmological argument and the Design argument. 
The Cosmological argument used Aristotle’s ideas on the scientific method and 
causation. Aquinas argued that there was a first cause, and he deduced that this first 
cause is God: 


1. Every effect has a cause. 

2. Nothing can cause itself. 

3. A causal chain cannot be of infinite length. 
4. Therefore there must be a first cause. 


The Antikythera [5] was an ancient mechanical device that is believed to have 
been designed to calculate astronomical positions. It was discovered in 1902 in a 
wreck off the Greek island of Antikythera, and dates from about 80 B.C. It is one of 
the oldest known geared devices, and it is believed that it was used for calculating 
the position of the sun, moon, stars and planets for a particular date entered. 

The Romans appear to have been aware of a device similar to the Antikythera 
that was capable of calculating the position of the planets. The island of Antikythera 
was well known in the Greek and Roman periods for its displays of mechanical 
engineering. 


?2Chrysippus was the head of the Stoics in the third century B.C. 
3 Aquinas’ (or St. Thomas’) most famous work is Summa Theologica. 
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Fig. 1.9 Julius Caesar 


1.5 The Romans 


Rome is said to have been founded** by Romulus and Remus about 750 B.C. Early 
Rome covered a small part of Italy, but it gradually expanded in size and impor- 
tance. It destroyed Carthage” in 146 B.C. to become the major power in the 
Mediterranean. The Romans colonized the Hellenistic world, and they were 
influenced by Greek culture and mathematics. Julius Caesar conquered the Gauls in 
58 B.C. (Fig. 1.9). 

The Gauls consisted of several disunited Celtic”® tribes. Vercingetorix succeeded 
in uniting them, but he was defeated at the siege of Alesia in 52 B.C. 


*4The Aeneid by Virgil suggests that the Romans were descended from survivors of the Trojan 
war, and that Aeneas brought surviving Trojans to Rome after the fall of Troy. 

Carthage was located in Tunisia, and the wars between Rome and Carthage are known as the 
Punic wars. Hannibal was one of the great Carthaginian military commanders, and during the 
second Punic war, he brought his army to Spain, marched through Spain and crossed the Pyrenees. 
He then marched along southern France and crossed the Alps into Northern Italy. His army also 
consisted of war elephants. Rome finally defeated Carthage and destroyed the city. 

©The Celtic period commenced around 1000 B.C. in Hallstatt (near Salzburg in Austria). The 
Celts were skilled in working with Iron and Bronze, and they gradually expanded into Europe. 
They eventually reached Britain and Ireland around 600 B.C. The early Celtic period was known 
as the “Hallstaat period” and the later Celtic period is known as “La Téne”. The later La Téne 
period is characterised by the quality of ornamentation produced. The Celtic museum in Hallein in 
Austria provides valuable information and artefacts on the Celtic period. The Celtic language 
would have similarities to the Irish language. However, the Celts did not employ writing, and the 
Ogham writing used in Ireland was developed in the early Christian period. 
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Fig. 1.10 Roman numbers 


I=1 
V=5 
xX=10 
L=50 
C= 100 
D=500 
M = 1000 


The Roman number system uses letters to represented numbers and a number 
consists of a sequence of letters. The evaluation rules specify that if a number 
follows a smaller number, then the smaller number is subtracted from the larger 
number: e.g. [IX represents 9 and XL represents 40. Similarly, if a smaller number 
followed a larger number, they were generally added: e.g. MCC represents 1200. 
They had no zero in their number system (Fig. 1.10). 

The use of Roman numerals was cumbersome in calculation, and an abacus was 
often employed. An abacus is a device that is usually of wood and has a frame that 
holds rods with freely sliding beads mounted on them. It is used as a tool to assist 
calculation, and it is useful for keeping track of the sums and the carries of 
calculations. 

It consists of several columns in which beads or pebbles are placed. Each col- 
umn represented powers of 10: i.e. 10°, 10', 107 and 10°. The column to the far 
right represents one, the column to the left 10, the next column to the left 100 and 
so on. Pebbles~’ (calculi) were placed in the columns to represent different num- 
bers: e.g. the number represented by an abacus with 4 pebbles on the far right; 2 
pebbles in the column to the left; and 3 pebbles in the next column to the left is 324. 
The calculations were performed by moving pebbles from column to column. 

Merchants introduced a set of weights and measures (including the libra for 
weights and the pes for lengths). They developed an early banking system to 
provide loans for businesses, and commenced minting money about 290 B.C. The 
Romans also made contributions to calendars, and Julius Caesar introduced the 
Julian calendar in 45 B.C. It has a regular year of 365 days divided into 12 months, 
and a leap day is added to February every 4 years. It remained in use up to the 
twentieth century, but has since been replaced by the Gregorian calendar. The 


Alphabet Symbol abcde fghij klmno pqrst uvwxyz 


Cipher Symbol dfegh ijklm nopgr stuvw xyzabe 


Fig. 1.11 Caesar Cipher 


"The origin of the word ‘Calculus’ is from Latin and means a small stone or pebble used for 
counting. 
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Fig. 1.12 Muhammad 
al-Khwarizmi 
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problem with the Julian calendar is that too many leap years are added over time. 
The Gregorian calendar was first introduced in 1582. 

The Romans employed the mathematics that had been developed by the Greeks. 
Caesar employed a substitution cipher on his military campaigns to enable 
important messages to be communicated safely. It involves the substitution of each 
letter in the plaintext (1.e. the original message) by a letter a fixed number of 
positions down in the alphabet. For example, a shift of 3 positions causes the letter 
B to be replaced by E, the letter C by F and so on. It is easily broken, as the 
frequency distribution of letters may be employed to determine the mapping 
(Fig. 1.11). The cipher is defined as 

The process of enciphering a message (i.e. plaintext) involves looking up each 
letter in the plaintext and writing down the corresponding cipher letter. The 
decryption involves the reverse operation: i.e. for each cipher letter the corre- 
sponding plaintext letter is identified from the table. 

The encryption may also be represented using modular arithmetic,** with the 
numbers 0-25 representing the alphabet letters, and addition (modulo 26) is used to 
perform the encryption. 


*8Modular arithmetic is discussed in chapter seven. 
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Fig. 1.13 Al Azhar University, Cairo 


The emperor Augustus”’ employed a similar substitution cipher (with a shift key 
of 1). The Caesar cipher remained in use up to the early twentieth century. How- 
ever, by then frequency analysis techniques were available to break the cipher. 


1.6 Islamic Influence 


Islamic mathematics refers to mathematics developed in the Islamic world from the 
birth of Islam in the early seventh century up until the seventeenth century. The 
Islamic world commenced with the Prophet Mohammed in Mecca, and spread 
throughout the Middle East, North Africa and Spain. The Golden Age of Islamic 
civilization was from 750 A.D. to 1250 A.D., and during this period enlightened 


°° Augustus was the first Roman emperor, and his reign ushered in a period of peace and stability 
following the bitter civil wars. He was the adopted son of Julius Caesar and was called Octavian 
before he became emperor. The earlier civil wars were between Caesar and Pompey, and following 
Caesar’s assassination civil war broke out between Mark Anthony and Octavian. Octavian 
defeated Anthony and Cleopatra at the battle of Actium, and became the first Roman emperor, 
Augustus. 
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caliphs recognized the value of knowledge, and sponsored scholars to come to 
Baghdad to gather and translate the existing world knowledge into Arabic. 

This led to the preservation of the Greek texts during the Dark Ages in Europe. 
Further, the Islamic cities of Baghdad, Cordoba and Cairo became key intellectual 
centres, and scholars added to existing knowledge (e.g. in mathematics, astronomy, 
medicine and philosophy), as well as translating the known knowledge into Arabic. 

The Islamic mathematicians and scholars were based in several countries in the 
Middle East, North Africa and Spain. Early work commenced in Baghdad, and the 
mathematicians were also influenced by the work of Hindu mathematicians who 
had introduced the decimal system and decimal numerals. Among the well-known 
Islamic scholars are Ibn al-Haytham, a tenth-century Iraqi scientist; Muhammad 
al-Khwarizmi (Fig. 1.12), the 9th Persian mathematician; Abd al-Rahman al-Sufi, a 
Persian astronomer who discovered the Andromeda galaxy; Ibn al-Nafis, a Syrian 
who did work on circulation in medicine; Averros, who was an Aristotelian 
philosopher from Cordoba in Spain; Avicenna who was a Persian philosopher; and 
Omar Khayyam who was a Persian Mathematician and poet. 

Many caliphs (Muslim rulers) were enlightened and encouraged scholarship in 
mathematics and science. They set up a centre for translation and research in 
Baghdad, and existing Greek texts such as the works of Euclid, Archimedes, 
Apollonius and Diophantus were translated into Arabic. Al-Khwarizmi made 
contributions to early classical algebra, and the word algebra comes from the Arabic 
word ‘al jabr’ that appears in a textbook by Al-Khwarizmi. The origin of the word 
algorithm is from the name of the Islamic scholar “Al-Khwarizmi”. 

Education was important during the Golden Age, and the Al Azhar University in 
Cairo (Fig. 1.13) was established in 970 A.D., and the Al-Qarawiyyin University in 
Fez, Morocco, was established in 859 A.D. The Islamic world has created beautiful 
architecture and art including the ninth-century Great Mosque of Samarra in Iraq; 
the tenth-century Great Mosque of Cordoba; and the eleventh-century Alhambra in 
Grenada. 

The Moors*” invaded Spain in the eighth century A.D., and they ruled large parts 
of the Peninsula for several centuries. Moorish Spain became a centre of learning, 
and this led to Islamic and other scholars coming to study at the universities in 
Spain. Many texts on Islamic mathematics were translated from Arabic into Latin, 
and these were invaluable in the renaissance in European learning and mathematics 
from the thirteenth century. The Moorish influence*’ in Spain continued until the 
time of the Catholic Monarchs* in the fifteenth century. Ferdinand and Isabella 
united Spain, defeated the Moors in Andalusia, and expelled them from Spain. 


3°The origin of the word ‘Moor’ is from the Greek work j1wopot, meaning very dark. It referred to 
the fact that many of the original Moors who came to Spain were from Egypt, Tunisia and other 
parts of North Africa. 

*!The Moorish influence includes the construction of various castles (alcazar), fortresses 
(alcalzaba) and mosques. One of the most striking Islamic sites in Spain is the palace of Alhambra 
in Granada, and it represents the zenith of Islamic art. 

??The Catholic Monarchs refer to Ferdinand of Aragon and Isabella of Castille who married in 
1469. They captured Granada (the last remaining part of Spain controlled by the Moors) in 1492. 
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The Islamic contribution to algebra was an advance on the achievements of the 
Greeks. They developed a broader theory that treated rational and irrational num- 
bers as algebraic objects, and moved away from the Greek concept of mathematics 
as being essentially Geometry. Later Islamic scholars applied algebra to arithmetic 
and geometry, and studied curves using equations. This included contributions to 
reduce geometric problems such as duplicating the cube to algebraic problems. 
Eventually, this led to the use of symbols in the fifteenth century such as 


Cag", 


The poet Omar Khayyam was also a mathematician who did work on the 
classification of cubic equations with geometric solutions. Other scholars made 
contributions to the theory of numbers: e.g. a theorem that allows pairs of amicable 
numbers to be found. Amicable numbers are two numbers such that each is the sum 
of the proper divisors of the other. They were aware of Wilson’s theory in number 
theory: i.e. for p prime then p divides (p — 1)!+1. 

The Islamic world was tolerant of other religious belief systems during the 
Golden Age, and there was freedom of expression provided that it did not infringe 
on the rights of others. It began to come to an end following the Mongol invasion 
and sack of Baghdad in the late 1250s and the Crusades. It continued to some extent 
until the conquest by Ferdinand and Isabella of Andalusia in the late fifteenth 
century. 


1.7 Chinese and Indian Mathematics 


The development of mathematics commenced in China about 1000 B.C. and was 
independent of developments in other countries. The emphasis was on 
problem-solving rather than on conducting formal proofs. It was concerned with 
finding the solution to practical problems such as the calendar, the prediction of the 
positions of the heavenly bodies, land measurement, conducting trade and the 
calculation of taxes. 

The Chinese employed counting boards as mechanical aids for calculation from 
the fourth century B.C. These are similar to abaci and are usually made of wood or 
metal, and contained carved grooves between which beads, pebbles or metal discs 
were moved. 

Early Chinese mathematics was written on bamboo strips and included work on 
arithmetic and astronomy. The Chinese method of learning and calculation in 
mathematics was learning by analogy. This involves a person acquiring knowledge 
from observation of how a problem is solved, and then applying this knowledge to 
solving similar kinds of problems. 

They had their version of Pythagoras’ theorem and applied it to practical 
problems. They were familiar with the Chinese remainder theorem, the formula for 
finding the area of a triangle, as well as showing how polynomial equations (up to 
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degree ten) could be solved. They showed how geometric problems could be solved 
by algebra, how roots of polynomials could be solved, how quadratic and simul- 
taneous equations could be solved, and how the area of various geometric shapes 
such as rectangles, trapezia and circles could be computed. Chinese mathematicians 
were familiar with the formula to calculate the volume of a sphere. The best 
approximation that the Chinese had to m was 3.14159, and this was obtained by 
approximations from inscribing regular polygons with 3 x 2” sides in a circle. 

The Chinese made contributions to number theory including the summation of 
arithmetic series and solving simultaneous congruences. The Chinese remainder 
theorem deals with finding the solutions to a set of simultaneous congruences in 
modular arithmetic. Chinese astronomers made accurate observations, which were 
used to produce a new calendar in the sixth century. This was known as the Taming 
Calendar and it was based on a cycle of 391 years. 

Indian mathematicians have made important contributions such as the devel- 
opment of the decimal notation for numbers that are now used throughout the 
world. This was developed in India sometime between 400 B.C. and 400 A.D. 
Indian mathematicians also invented zero and negative numbers, and also did early 
work on the trigonometric functions of sine and cosine. The knowledge of the 
decimal numerals reached Europe through Arabic mathematicians, and the resulting 
system is known as the Hindu—Arabic numeral system. 

The Sulva Sutras is a Hindu text that documents Indian mathematics, and it dates 
from about 400 B.C. The Indians were familiar with the statement and proof of 
Pythagoras’ theorem, rational numbers, quadratic equations, as well as the calcu- 
lation of the square root of 2 to five decimal places. 


1.8 Review Questions 


Discuss the strengths and weaknesses of the various numbering system. 
. Describe the ciphers used during the Roman civilization and write a 
program to implement one of these. 

Discuss the nature of an algorithm and its importance in computing. 
Discuss the working of an abacus and its application to calculation. 
What are the differences between syllogistic logic and stoic logic? 
Describe the main achievements of the Islamic world in mathematics. 
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1.9 Summary 


Software is pervasive in the modern world, and it has transformed the world in 
which we live. New technology has led to improvements in all aspects of our lives 
including medicine, transport, education and so on. The pace of change of new 
technology is relentless, with new versions of technology products becoming 
available several times a year. 

This chapter considered some of the contributions of early civilizations to 
computing. We commenced our journey with an examination of some of the 
contributions of the Babylonians. We then moved forward to consider some of the 
achievements of the Egyptians, the Greeks and Romans; Islamic scholars; and the 
Indians and Chinese. 

The Babylonians recorded their mathematical knowledge on clay cuneiform 
tablets. These tablets included tables for multiplication, division, squares, and square 
roots and the calculation of area. They were familiar with techniques that allowed the 
solution of a linear equation and one root of a quadratic equation to be determined. 

The Egyptian civilization developed along the River Nile, and they applied their 
knowledge of mathematics to solve practical problems such as measuring the 
annual Nile flooding, and constructing temples and pyramids. 

The Greeks and the later Hellenistic period made important contributions to 
Western civilization. Their contributions to mathematics included the Euclidean 
algorithm, which is used to determine the greatest common divisor of two numbers. 
Eratosthenes developed an algorithm to determine the prime numbers up to a given 
number. Archimedes invented the “Archimedes Screw”, the “Archimedes Claw” 
and a type of heat ray. 

The Islamic civilization helped to preserve Western knowledge that was lost 
during the Dark Ages in Europe, and they also continued to develop mathematics 
and algebra. Hindu mathematicians introduced the decimal notation that is familiar 
today. Islamic mathematicians adopted it and the resulting system is known as the 
Hindu—Arabic system. 
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Sets, Relations and Functions 


2.1. Introduction 


This chapter provides an introduction to fundamental building blocks in mathe- 
matics such as sets, relations and functions. Sets are collections of well-defined 
objects; relations indicate relationships between members of two sets A and B; 
functions are a special type of relation where there is exactly (or at most)! one 
relationship for each element a € A with an element in B. 

A set is a collection of well-defined objects that contains no duplicates. The term 
“well defined’ means that for a given value it is possible to determine whether or not 
it is a member of the set. There are many examples of sets such as the set of natural 
numbers N, the set of integer numbers Z and the set of rational numbers Q. The 
natural numbers N is an infinite set consisting of the numbers {1, 2, ....}. Venn 
diagrams may be used to represent sets pictorially. 

A binary relation R(A, B) where A and B are sets is a subset of the Cartesian 
product (A x B) of A and B. The domain of the relation is A and the codomain of 
the relation is B. The notation aRb signifies that there is a relation between a and 
b and that (a,b)€R. An n-ary relation R(A,,A2,...A,) is a subset of 
(A, x Ap X --+ X A,). However, an n-ary relation may also be regarded as a binary 
relation R(A,B) with A= A, x A, x --- x A,_; and B= A,. 

Functions may be total or partial. A total function f : A — B is a special relation 
such that for each element a € A, there is exactly one element b € B. This is 
written as f(a) = b. A partial function differs from a total function in that the 
function may be undefined for one or more values of A. The domain of a function 


'We distinguish between total and partial functions. A total function f : A > B is defined for every 
element in A, whereas a partial function may be undefined for one or more values in A. 
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(denoted by dom /) is the set of values in A for which the partial function is defined. 
The domain of the function is A provided that fis a total function. The codomain of 
the function is B. 


2.2 Set Theory 


A set is a fundamental building block in mathematics, and it is defined as a col- 
lection of well-defined objects. The elements in a set are of the same kind, and they 
are distinct with no repetition of the same element in the set.* Most sets encountered 
in computer science are finite, as computers can only deal with finite entities. Venn 
diagrams* are often employed to give a pictorial representation of a set, and they 
may be used to illustrate various set operations such as set union, intersection and 
set difference. 

There are many well-known examples of sets including the set of natural 
numbers denoted by N, the set of integers denoted by Z, the set of rational numbers 
denoted by Q, the set of real numbers denoted by R and the set of complex numbers 
denoted by C. 


Example 2.1 The following are examples of sets. 


— The books on the shelves in a library; 

— The books that are currently overdue from the library; 

— The customers of a bank; 

— The bank accounts in a bank; 

— The set of Natural Numbers N = {1, 2, 3, ...}; 

— The Integer Numbers Z = {..., —3, —2, -1, 0, 1, 2, 3, ...}; 
— The non-negative integers Z* = {0, 1, 2, 3, ...}; 

— The set of Prime Numbers = {2, 3, 5, 7, 11, 13, 17, ...}; 
— The Rational Numbers is the set of quotients of integers. 


Q= {/,:7,4 © Zandgq F O}. 


There are mathematical objects known as multi-sets or bags that allow duplication of elements. 
For example, a bag of marbles may contain three green marbles, two blue and one red marble. 
>The British logician, John Venn, invented the Venn diagram. It provides a visual representation of 
a set and the various set theoretical operations. Their use is limited to the representation of two or 
three sets as they become cumbersome with a larger number of sets. 
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A finite set may be defined by listing all of its elements. For example, the set 
A = {2, 4, 6, 8, 10} is the set of all even natural numbers less than or equal to 10. 
The order in which the elements are listed is not relevant: i.e. the set {2, 4, 6, 8, 10} 
is the same as the set {8, 4, 2, 10, 6}. 


G)" 


Sets may be defined by using a predicate to constrain set membership. For 
example, the set S = {n : N:n<10Anmod2 = 0} also represents the set {2, 4, 6, 
8, 10}. That is, the use of a predicate allows a new set to be created from an existing 
set by using the predicate to restrict membership of the set. The set of even natural 
numbers may be defined by a predicate over the set of natural numbers that restricts 
membership to the even numbers. It is defined by 


Evens = {x|x € NA even(x)}. 


In this example, even(x) is a predicate that is true if x is even and false otherwise. 
In general, A = {x € E|P(x)} denotes a set A formed from a set E using the 
predicate P to restrict membership of A to those elements of E for which the 
predicate is true. 

The elements of a finite set S are denoted by {x,, x2, ... x,}. The expression 
x € S denotes that the element x is a member of the set S, whereas the expression 
x ¢ S indicates that x is not a member of the set S. 


A set S is a subset of a set T (denoted by SCT) if whenever s € S then s € T, 
and in this case the set T is said to be a superset of S (denoted by T > S':). Two sets 
S and T are said to be equal if they contain identical elements: i.e. S = T if and only 
if SCT and T CS. A set S is a proper subset of a set T (denoted by S C T) if SCT 
and S # T. That is, every element of S is an element of T, and there is at least one 
element in T that is not an element of S. In this case, T is a proper superset of 
S (denoted by T D S). 

The empty set (denoted by @ or {}) represents the set that has no elements. 
Clearly, 9 is a subset of every set. The singleton set containing just one element x is 
denoted by {x}, and clearly x € {x} and x 4 {x}. Clearly, y € {x} if and only if 
x=y. 


Example 2.2 


G@) {1,2} C{1,2,3}. 
Gi) OCNCZCQCRCC. 
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The cardinality (or size) of a finite set S defines the number of elements present in 
the set. It is denoted by |S]. The cardinality of an infinite’ set S is written as 
|S| =o. 


Example 2.3 


(i) Given A = {2,4,5,8, 10} then |A| =5. 
(ii) GivenA = {x € Z: x? = 9} then |A| = 2. 
(iii) Given A = {x € Z: x? = —9} then |A| = 0. 


2.2.1 Set Theoretical Operations 


Several set theoretical operations are considered in this section. These include the 
Cartesian product operation; the power set of a set; the set union operation; the set 
intersection operation; the set difference operation; and the symmetric difference 
operation. 


Cartesian Product 


The Cartesian product allows a new set to be created from existing sets. The 
Cartesian° product of two sets S and T (denoted by S x T) is the set of ordered pairs 
{(s,2)|s € S,t © T}. Clearly, S x T #T x S and so the Cartesian product of two 
sets is not commutative. Two ordered pairs (s1,f,) and (s2, t2) are considered equal if 
and only if s; = sz and t, = fh. 

The Cartesian product may be extended to that of n sets S$), So, ..., S,. The 
Cartesian product S$, x S;x...x S, is the set of ordered tuples 
{(51, 52, --; Sn) [51 € Si, 52 © So, .., 5, € Sn}. Two ordered n-tuples (51, 52, ..., Sn) 
and (sj’, So’, ..., Sy’) are considered equal if and only if s; = 5)’, so, = 5’, ..., 
S, = 5; 

The Cartesian product may also be applied to a single set S to create ordered n- 
tuples of S: ie. S’ = S x S x .. x S (n times). 


Power Set 


The power set of a set A (denoted by PA) denotes the set of subsets of A. For 
example, the power set of the set A = {1, 2, 3} has 8 elements and is given by 


A= {0, {1}, {2}, {3}, {1,2}, {1,3}, {2,3}, {1,2,3}}. 


“The natural numbers, integers and rational numbers are countable sets, whereas the real and 
complex numbers are uncountable sets. 

Cartesian product is named after René Descartes who was a famous 17th French mathematician 
and philosopher. He invented the Cartesian coordinate system that links geometry and algebra, and 
allows geometric shapes to be defined by algebraic equations. 
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There are 2° = 8 elements in the power set of A = {1, 2, 3} and the cardinality of 
A is 3. In general, there are 2/4I elements in the power set of A. 


Theorem 2.1 (Cardinality of Power Set of A) 
There are 2|A| elements in the power set of A. 


Proof Let |A| =n, then the cardinality of the subsets of A are subsets of size 0,1, 


k 
of A is the total number of subsets of size 0,1,2, ... up to n. That is, 


"-£(0) 


k=1 


..../. There are ( ‘) subsets of A of size k.° Therefore, the total number of subsets 


The Binomial theorem (we prove it in Example 4.2 in Chap. 4) states that 
n_ . n 
(ays oe 


Therefore, putting x = 1, we get 


Union and Intersection Operations 


The union of two sets A and B is denoted by A UB. It results in a set that contains all 
of the members of A and of B and is defined by 


AUB = {rr € Aorr € B}. 


For example, suppose A = {1, 2, 3} and B = {2, 3, 4} then AUB = {1,2,3, 4}. 
Set union is a commutative operation: i.e. AUB = BUA. Venn diagrams are used 
to illustrate these operations pictorially. 


A B A B 
AUB ANB 


The intersection of two sets A and B is denoted by ANMB. It results in a set 
containing the elements that A and B have in common and is defined by 


We discuss permutations and combinations in Chap. 5. 
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ANB = {r|r € A and r € B}. 


Suppose A = {1, 2,3} and B = {2, 3, 4}, then ANB = {2,3}. Set intersection is 
a commutative operation: ie. AN B= BNA. 

Union and intersection are binary operations but may be extended to more 
generalized union and intersection operations. For example: 


in : 
U., A; denotes the union of n sets. 


ee A; denotes the intersection of 7 sets. 


Set Difference Operations 


The set difference operation A\B yields the elements in A that are not in B. It is 
defined by 


A\B = {ala € Aanda ¢ B}. 


For A and B defined as A = {1, 2} and B = {2, 3}, we have A\B = {1} and B 
\A = {3}. Clearly, set difference is not commutative: i.e. A\B # B\A. Clearly, 
A\A = @ and A\@ =A. 

The symmetric difference of two sets A and B is denoted by A AB and is given 
by 


AAB=A\BUB\A. 


The symmetric difference operation is commutative: ie. AA B= BAA. Venn 
diagrams are used to illustrate these operations pictorially. 


A B A B A B 
A\B B\A AAB 


The complement of a set A (with respect to the universal set U) is the elements in 
the universal set that are not in A. It is denoted by A‘ (or A’) and is defined as 


A‘ = {ulu € Uandu ¢ A} = U\A. 


The complement of set A is illustrated by the shaded area below. 


U 


(+) 7 — Ac 
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Table. 2.1 Properties of set operations 


Property Description 

Commutative Union and intersection operations are commutative: 
SUT=TUS 
SOT=TNS 

Associative Union and intersection operations are associative: 


RU(SUT) = (RUS) UT 
RO(SNT) = (RNS) AT 
Identity The identity under set union is the empty set 0, and the identity under 
intersection is the universal set U 
SUO=9US=S 
SNU=UNS=S 
Distributive The union operator distributes over the intersection operator and vice versa 
RA(SUT) = (RNS) U(RNT) 
RU(SNT) = (RUS) (RUT) 
De Morgan’s’ The complement of SUT is given by 
Law (SUIT = Sinz 
The complement of SMT is given by 
(SNM, = Une 


2.2.2 Properties of Set Theoretical Operations 
The set union and set intersection properties are commutative and associative. Their 
properties are listed in Table 2.1. 


These properties may be seen to be true with Venn diagrams, and we give proof of 
the distributive property (this proof uses logic which is discussed in Chaps. 14—16). 


Proof of Properties (Distributive Property) 


To show AN (BUC) = (ANB) U(ANC), 
suppose x € AN (BUC) then 


x€AAx€ (BUC) 
=>xE€AA(XEBVXEC) 
=> (x EC AAXEB)V(xEAAXEC) 
=>x€(ANB)Vxe€ (ANC) 


=> x € (ANB)U(ANC). 


"De Morgan’s law is named after Augustus De Morgan, a nineteenth century English 
mathematician who was a contemporary of George Boole. 
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Therefore, AN (BUC) C (ANB) U(ANC). 
Similarly, (AM B)U(ANC) CAN(BUC). 
Therefore, AN (BUC) = (ANB)U(ANC). 


2.2.3 Russell’s Paradox 


Bertrand Russell (Fig. 2.1) was a famous British logician, mathematician and 
philosopher. He was the co-author with Alfred Whitehead of Principia Mathemat- 
ica, which aimed to derive all of the truths of mathematics from logic. Russell’s 
paradox was discovered by Bertrand Russell in 1901, and showed that the system of 
logicism being proposed by Frege (discussed in Chap. 14) contained a contradiction. 


QUESTION (POSED BY RUSSELL TO FREGE) 
Is the set of all sets that do not contain themselves as members a set? 
RUSSELL’S PARADOX 


Let A={S a set and S ¢ S}. Is AG A? Then ACAS>A EA and vice versa. 
Therefore, a contradiction arises in either case and there is no such set A. 

Two ways of avoiding the paradox were developed in 1908, and these were 
Russell’s theory of types and Zermelo set theory. Russell’s theory of types was a 
response to the paradox by arguing that the set of all sets is ill-formed. Russell 
developed a hierarchy with individual elements at the lowest level, sets of elements 
at the next level, sets of sets of elements at the next level and so on. It is then 
prohibited for a set to contain members of different types. 

A set of elements has a different type from its elements, and one cannot speak of the 
set of all sets that do not contain themselves as members as these are of different types. 
The other way of avoiding the paradox was Zermelo’s axiomatization of set theory. 


Fig. 2.1 Bertrand Russell 
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Remark Russell’s paradox may also be illustrated by the story of a town that has 
exactly one barber who is male. The barber shaves all and only those men in town 
who do not shave themselves. The question is who shaves the barber. 


If the barber does not shave himself, then according to the rule he is shaved by 
the barber (i.e. himself). If he shaves himself, then according to the rule he is not 
shaved by the barber (i.e. himself). 

The paradox occurs due to self-reference in the statement, and a logical exam- 
ination shows that the statement is a contradiction. 


2.2.4 Computer Representation of Sets 


Sets are fundamental building blocks in mathematics, and so the question arises as 
to how a set is stored and manipulated in a computer. The representation of a set 
M on a computer requires a change from the normal view that the order of the 
elements of the set is irrelevant, and we will need to assume a definite order in the 
underlying universal set .4 from which the set M is defined. 

That is, a set is always defined in a computer program with respect to an 
underlying universal set, and the elements in the universal set are listed in a definite 
order. Any set M arising in the program that is defined with respect to this universal 
set .& is a subset of .H. Next, we show how the set M is stored internally on the 
computer. 

The set M is represented in a computer as a string of binary digits b,b>...b, 
where n is the cardinality of the universal set .W%. The bits b; (where i ranges over 
the values 1, 2, ... m) are determined according to the rule: 


b; = lif ithelementof isin M; 
b; = Oif ithelement of is notinM. 


For example, if. H = {1, 2, ... 10} then the representation of M = {1, 2,5, 8} is 
given by the bit string 1100100100 where this is given by looking at each element 
of .#@ in turn and writing down 1 if it is in M and 0 otherwise. 

Similarly, the bit string 0,100,101,100 represents the set M = {2, 5, 7, 8}, and 
this is determined by writing down the corresponding element in .@ that corre- 
sponds to a | in the bit string. 

Clearly, there is a one-to-one correspondence between the subsets of ./% and all 
possible n-bit strings. Further, the set theoretical operations of set union, intersec- 
tion and complement can be carried out directly with the bit strings (provided that 
the sets involved are defined with respect to the same universal set). This involves a 
bitwise ‘or’ operation for set union, a bitwise ‘and’ operation for set intersection 
and a bitwise ‘not’ operation for the set complement operation. 
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2.3 Relations 


A binary relation R(A, B) where A and B are sets is a subset of A x B: ie. 
RCA x B. The domain of the relation is A and the codomain of the relation is 
B. The notation aRb signifies that (a, b) € R. 

A binary relation R(A, A) is a relation between A and A. This type of relation 
may always be composed of itself, and its inverse is also a binary relation on A. The 
identity relation on A is defined by a iaa for all a € A. 


Example 2.4 There are many examples of relations: 


(i) The relation on a set of students in a class where (a, b) € R if the height of a is 
greater than the height of b. 

(ii) The relation between A and B where A = {0, 1, 2} and B= {3, 4, 5} with 
R given by 


R= {(0, 3), (0,4), (1, 4)}. 


(iii) The relation less than (<) between and R and R is given by 


{(x,y) € R? :x<y}. 


(iv) A bank may represent the relationship between the set of accounts and the set 
of customers by a relation. The implementation of a bank account may often be 
a positive integer with at most eight decimal digits. 

The relationship between accounts and customers may be done with a 
relation RCA x B, with the set A chosen to be the set of natural numbers, and 
the set B chosen to be the set of all human beings alive or dead. The set A could 
also be chosen to be A = {n EN: n< 108}. 


A relation R(A, B) may be represented pictorially. This is referred to as the graph 
of the relation, and it is illustrated in the diagram below. An arrow from x to y is 
drawn if (x, y) is in the relation. Thus, for the height relation R given by {(a, p), (a, 
r), (b, q)} an arrow is drawn from a to p, from a to r and from b to gq to indicate that 
(a, p), (a, r) and (b, q) are in the relation R. 
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The pictorial representation of the relation makes it easy to see that the height of 
a is greater than the height of p and r, and that the height of D is greater than the 
height of gq. 

An n-ary relation R(A;,A2,...An) is a subset of (Aj x Az x --- x Ay). How- 
ever, an n-ary relation may also be regarded as a binary relation R(A, B) with 
A= Ay, X Az X°+:: X Ay_; and B= Ay. 


2.3.1 Reflexive, Symmetric and Transitive Relations 


There are various types of relations including reflexive, symmetric and transitive 
relations. 


i) A relation on a set A is reflexive if (a, a) € R for alla € A. 
(i) A relation R is symmetric if whenever (a, b) € R then (b, a) € R. 
(iii) A relation is transitive if whenever (a, b) € R and (b, c) € R then (a, c) € R. 


A relation that is reflexive, symmetric and transitive is termed an equivalence 
relation. 


Example 2.5 (Reflexive Relation) 
A relation is reflexive if each element possesses an edge looping around on itself. 
The relation in Fig. 2.2 below is reflexive. 


Example 2.6 (Symmetric Relation) 

The graph of a symmetric relation will show for every arrow from a to b an 
opposite arrow from b to a. The relation in Fig. 2.3 is symmetric: i.e. whenever (a, 
b) € R then (b, a) € R. 


Example 2.7 (Transitive relation) 

The graph of a transitive relation will show that whenever there is an arrow from 
a to b and an arrow from Db to c that there is an arrow from a to c. The relation in 
Fig. 2.4 below is transitive: i.e. whenever (a, b) € R and (b, c) € R then (a, c) € R. 


Fig. 2.2 Reflexive relation 
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Fig. 2.3 Symmetric relation 


Fig. 2.4 Transitive relation 


Example 2.8 (Equivalence relation) 

The relation on the set of integers Z defined by (a, b) € R if a — b = 2k for some 
k € Z is an equivalence relation, and it partitions the set of integers into two 
equivalence classes: i.e. the even and odd integers. 


Domain and Range of Relation 


The domain of a relation R (A, B) is given by {a € A| db € Band (a,b) € R}. It is 
denoted by dom R. The domain of the relation 
R= {(4,P), (a ? ts (b,q)} is {a,b}. 

The range of a relation R (A, B) is given by {b € B| 5a € Aand (a,b) € R}. It is 
denoted by rng R. The range of the relation R = {(a,p), (a,r), (b, q)} is {p, q, r}. 


Inverse of a Relation 


Suppose RCA x B is a relation between A and B, then the inverse relation 
R~' CB x A is defined as the relation between B and A and is given by 


bR ‘aif and only if aRb. 
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That is, 


R''={(b,a)€BxA: (a,b) €R}. 


Example 2.9 Let R be the relation between Z and Z* defined by mRn if and only if 
m? =n. Then R= {(m,n) € Z x Z* : mm? =n} and 
R= {(n,m) € Zt x Z: ml =n}. 


For example, —3 R 9, —4 R 16,0 R 0, 16 R'' —4, 9 R! -3, ete. 
Partitions and Equivalence Relations 


An equivalence relation on A leads to a partition of A, and vice versa for every 
partition of A there is a corresponding equivalence relation. 

Let A be a finite set and let Aj, Ao, ..., A, be subsets of A such that c Ai 4 @) for 
all i, A; A; = 0 if i A j and A = U7A; = Ay UAQU...U Ay. 

The sets A,partition the set A, and these sets are called the classes of the partition 
(Fig. 2.5). 


Theorem 2.2 (Equivalence Relation and Partitions) 


An equivalence relation on A gives rise to a partition of A where the equivalence 
classes are given by Class(a) = {x|x € A and (a,x) € R}. Similarly, a partition 
gives rise to an equivalence relation R, where (a, b) € R if and only if a and b are in 
the same partition. 


Proof Clearly, a € Class(a) since R is reflexive and clearly the union of the 
equivalence classes is A. Next, we show that two equivalence classes are either 
equal or disjoint. 


Suppose Class (a) M Class (b) 4 9. Let x € Class(a) M Class(b) and so (a, 
x) and (b, x) € R. By the symmetric property (x, b) € R and since R is transitive 
from (a, x) and (x, b) in R, we deduce that (a, b) € R. Therefore b € Class(a). 
Suppose y is an arbitrary member of Class (b) then (b, y) € R therefore from (a, 
b) and (b, y) in R, we deduce that (a, y) is in R. Therefore, since y was an arbitrary 
member of Class(a), we deduce that Class(bCClass(a). Similarly, 
Class (a) C Class (b) and so Class(a) = Class(b). 


Fig. 2.5 Partitions of A 
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This proves the first part of the theorem and for the second part, we define a 
relation R such that (a, b) € Rif a and bare in the same partition. It is clear that this 
is an equivalence relation. 


2.3.2. Composition of Relations 


The composition of two relations R\(A, B) and R2(B, C) is given by Rz 0 R,; where 
(a, c) € Ro o R, if and only if there exists b € B such that (a, b) € R,; and (b, 
c) € R>. The composition of relations is associative: i.e. 


(R3 oR?) oR, = R3 0(Ro oR). 


Example 2.10 (Composition of Relations) 

Consider a library that maintains two files. The first file maintains the serial number 
s of each book as well as the details of the author a of the book. This may be 
represented by the relation R;= sR,a. The second file maintains the library card 
number c of its borrowers and the serial number s of any books that they have 
borrowed. This may be represented by the relation Rp = cRos. 


The library wishes to issue a reminder to its borrowers of the authors of all books 
currently on loan to them. This may be determined by the composition of R,; 0 R3: 
i.e. c Ryo R> a if there is a book with serial number s such that c Rp s and s R, a. 


Example 2.11 (Composition of Relations) 

Consider sets A = {a, b, c}, B = {d, e, f}, C = {g, h, i} and relations R(A, B) = {(a, 
d), (a, f), (b, d), (c, e)} and S(B, C) = {(d, h), (d, 0), (e, g), (e, h)}. Then we graph 
these relations and show how to determine the composition pictorially. 


So R is determined by choosing x € A and y € C and checking if there is a route 
from x to y in the graph (Fig. 2.6). If so, we join x to y in S o R. For example, if we 
consider a and h we see that there is a path from a to d and from d to h, and 
therefore (a, /) is in the composition of S and R. 


The union of two relations R,(A, B) and R2(A, B) is meaningful (as these are both 
subsets of A x B). The union R; UR; is defined as (a,b) € R; UR> if and only if 
(a, b) € R, or (a, b) € Ro. 
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R(A,B) S(B,C) 


Fig. 2.6 Composition of relations 


Similarly, the intersection of R; and R2(R; 1 R2) is meaningful and is defined as 
(a,b) € Ri NR> if and only if (a, b) € R, and (a, b) € R». The relation R; is a 
subset of R2(R; C R2) if whenever (a, b) € R, then (a, b) € Ro. 

The inverse of the relation R was discussed earlier and is given by the relation 
R | where R~! = {(b,a)|(a,b) € R}. 

The composition of R and R | yields R~'o R = {(a,a)|a € dom R} = ia and 
RoR! = {(b,b)|b € domR™'} = ig. 


2.3.3 Binary Relations 
A binary relation R on A is a relation between A and A, and a binary relation can 
always be composed of itself. Its inverse is a binary relation on the same set. The 
following are all relations on A: 
R°=RoR 
R? =(RoR) oR 
R° = ia (identity relation) 
SR ok. 


Example 2.12 Let R be the binary relation on the set of all people P such that 
(a, b) € R if a is a parent of b. Then the relation R” is interpreted as 


R is the parent relationship. 

R° is the grandparent relationship. 

R? is the great grandparent relationship. 
R'' is the child relationship. 
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R * is the grandchild relationship. 

R° is the great grandchild relationship. 

This can be generalized to a relation R” on A where R"=RoORO... 0 R (n- 
times). The transitive closure of the relation R on A is given by 


r= cue = RoUR'UR?U...R"U.. 


where R° is the reflexive relation containing only each element in the domain of R: 
ie. R° =i, = {(a,a)|a € domR}. 

The positive transitive closure is similar to the transitive closure except that it 
does not contain R°. It is given by 


R*+ =| [° R'=R'UR*U...UR"U.. 


i=1 


a R* b if and only if a R” b for some n > 0: i.e. there exists c1, Cy ... C, € A such 
that 


aRc,, Cc, Rc2,...,CnRb 


Parnas® introduced the concept of the limited domain relation (LD relation), and 
an LD relation L consists of an ordered pair (R;, C,) where R; is a relation and C,, is 
a subset of Dom R_. The relation Ry is on a set U and Cy; is termed the competence 
set of the LD relation L. A description of LD relations and a discussion of their 
properties are in Chap. 2 of [1]. 

The importance of LD relations is that they may be used to describe program 
execution. The relation component of the LD relation L describes a set of states 
such that if execution starts in state x, it may terminate in state y. The set U is the set 
of states. The competence set of L is such that if execution starts in a state that is in 
the competence set, then it is guaranteed to terminate. 


2.3.4 Applications of Relations 


A Relational Database Management System (RDBMS) is a system that manages 
data using the relational model, and examples of such systems include RDMS 
developed at MIT in the 1970s; Ingres developed at the University of California, 
Berkeley, in the mid-1970s; Oracle developed in the late 1970s; DB2; Informix; 
and Microsoft SQL Server. 

A relation is defined as a set of tuples and is usually represented by a table. 
A table is data organized in rows and columns, with the data in each column of the 
table of the same data type. Constraints may be employed to provide restrictions on 


Parnas made important contributions to software engineering in the 1970s. He invented 
information hiding which is used in object-oriented design. 
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the kinds of data that may be stored in the relations. Constraints are Boolean 
expressions which indicate whether the constraint holds or not, and are a way of 
implementing business rules in the database. 

Relations have one or more keys associated with them, and the key uniquely 
identifies the row of the table. An index is a way of providing fast access to the data 
in a relational database, as it allows the tuple in a relation to be looked up directly 
(using the index) rather than checking all of the tuples in the relation. 

The Structured Query Language (SQL) is a computer language that tells the 
relational database what to retrieve and how to display it. A stored procedure is 
executable code that is associated with the database, and it is used to perform 
common operations on the database. 

The concept of a relational database was first described in a paper “A Relational 
Model of Data for Large Shared Data Banks” by Codd [2]. A relational database is 
a database that conforms to the relational model, and it may be defined as a set of 
relations (or tables). 

Codd (Fig. 2.7) developed the relational database model in the late 1960s, and 
today, this is the standard way that information is organized and retrieved from 
computers. Relational databases are at the heart of systems from hospitals’ patient 
records to airline flight and schedule information. 

A binary relation R(A, B) where A and B are sets is a subset of the Cartesian 
product (A x B) of A and B. The domain of the relation is A, and the codomain of 
the relation is B. The notation aRb signifies that there is a relation between a and 
b and that (a4, b) € R. An m-ary relation R(A;, A2,...An) is a subset of the 
Cartesian product of the n sets: i.e. a subset of (A; x Ao x --- x Ay). However, an 
n-ary relation may also be regarded as a binary relation R(A, B) with A= 
A, X Ag X--- x A,_; and B = A,,. 

The data in the relational model are represented as a mathematical n-ary relation. 
In other words, a relation is defined as a set of n-tuples, and is usually represented 
by a table. A table is a visual representation of the relation, and the data is organized 


Fig. 2.7 Edgar Codd 
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Weight 
Pl Nut Red 12 London 
RD Bolt Green 17 Paris 
P3 Screw Blue 17 Rome 
P4 Screw Red 14 London 
PS Cam Blue 12 Paris 


Fig. 2.8 PART relation 


in rows and columns. The data stored in each column of the table are of the same 
data type. 

The basic relational building block is the domain or data type (often called just 
type). Each row of the table represents one n-tuple (one tuple) of the relation, and 
the number of tuples in the relation is the cardinality of the relation. Consider the 
PART relation taken from [3], where this relation consists of a heading and the 
body. There are five data types representing part numbers, part names, part colours, 
part weights and locations in which the parts are stored. The body consists of a set 
of n-tuples, and the PART relation given in Fig. 2.8 is of cardinality six. 

For more information on the relational model and databases, see [4]. 


2.4 Functions 


A function f : A — B is a special relation such that for each element a € A, there is 
exactly (or at most)” one element b € B. This is written as f(a) = b. 


A function is a relation but not every relation is a function. For example, the 
relation in the diagram below is not a function since there are two arrows from the 
element a € A. 


°We distinguish between total and partial functions. A total function is defined for all elements in 
the domain, whereas a partial function may be undefined for one or more elements in the domain. 
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Fig. 2.9 Domain and range 
of a partial function 


The domain of the function (denoted by dom f/f) is the set of values in A for 
which the function is defined. The domain of the function is A provided that fis a 
total function. The codomain of the function is B. The range of the function 
(denoted by rng f) is a subset of the codomain and consists of 


mgf = {r|r € Bsuch that f(a) = rforsomea € A}. 


Functions may be partial or total. A partial function (or partial mapping) may be 
undefined for some values of A, and partial functions arise regularly in the com- 
puting field (Fig. 2.9). Total functions are defined for every value in A and many 
functions encountered in mathematics are total. 


Example 2.13 (Functions) 

Functions are an essential part of mathematics and computer science, and there are 
many well-known functions such as the trigonometric functions sin(x), cos(x) and 
tan(x); the logarithmic function In(x); the exponential functions e*; and polynomial 
functions. 


(i) Consider the partial function f : R — R where 


f(x) =1/x (wherex # 0). 


This partial function is defined everywhere except for x = 0. 


(ii) Consider the function f : R — R where 


f(x) =x. 


Then this function is defined for all x € R. 
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Partial functions often arise in computing as a program may be undefined or fail 
to terminate for several values of its arguments (e.g. infinite loops). Care is required 
to ensure that the partial function is defined for the argument to which it is to be 
applied. 

Consider a program P that has one natural number as its input and which for 
some input values will never terminate. Suppose that if it terminates, it prints a 
single real result and halts. Then P can be regarded as a partial mapping from N to 
R. 


P:N—-R. 


Example 2.14 How many total functions f : A — B are there from A to B (where 
A and B are finite sets)? 


Each element of A maps to any element of B, i.e. there are |B] choices for each 
element a € A. Since there are |A| elements in A, the number of total functions is 
given by 


|B] |B|.....|B] ({A| times) 
= |B|“ total functions between A and B. 


Example 2.15 How many partial functions f : A — B are there from A to B (where 
A and B are finite sets)? 


Each element of A may map to any element of B or to no element of B (as it may 
be undefined for that element of A). In other words, there are |B|+1 choices for each 
element of A. As there are |A| elements in A, the number of distinct partial functions 
between A and B is given by 


(|B| + 1) (JB|+1).....(B]+1)  (|A| times) 
= ([B|+ 1)! 
Two partial functions f and g are equal if 


1. dom f= dom g; 
2. fla) = g(a) for all a € dom f 


A function fis less defined than a function g(f C g) if the domain of fis a subset 
of the domain of g, and the functions agree for every value on the domain of f: 


1. domf Cdom g; 
2. fla) = g(a) for all a € dom f. 
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The composition of functions is similar to the composition of relations. Suppose 
f:A— Band g:B-— C then gof : A — Cis a function, and this is written as g o 
SX) or g(f(x)) for x € A. 

The composition of functions is not commutative and this can be seen by an 
example. Consider the function f : R — R such that f(x) = x? and the function g: 
R — R such that g(x) = x+2. Then 


gof(x) = g(x?) =x +2. 


fog(x) =f(x+2) = (x+2)°= vr +4x+4, 


Clearly, g of(x) 4 f o g(x) and so composition of functions is not commutative. 
The composition of functions is associative, as the composition of relations is 
associative and every function is a relation. For f: A—B, g: B—C and 
h: C—D, we have 


ho(gof) = (hog) of. 


A function f : A — B is injective (one to one) if 


f(a) =f(a) > a1 = a. 


For example, consider the function f:R—R_ with f(x) =x°. Then 
Ff GB) =f (-3) = 9, and so this function is not one to one. 

A function f(x) = x is surjective (onto) if given any b € B there exists an 
a € A such that f(a) = b. Consider the function f :R—R with f(x) =x+1. 
Clearly, given any r € R, then f (7-1) = r and so f is onto (Fig. 2.10). 

A function is bijective if it is one to one and onto. That is, there is a one-to-one 
correspondence between the elements in A and B, and for each b € B there is a 
unique a € A such that f(a) = b (Fig. 2.11). 

The inverse of a relation was discussed earlier and the relational inverse of a 
function f : A — B clearly exists. The relational inverse of the function may or may 
not be a function. 


4) _ 


1-1, Not Onto Onto, Not 1-1 


Fig. 2.10 Injective and surjective functions 
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Fig. 2.11 Bijective function (One to one and Onto) 


However, if the relational inverse is a function, it is denoted by f-!: B — A. 
A total function has an inverse if and only if it is bijective, whereas a partial 
function has an inverse if and only if it is injective. 

The identity function 1, : A — A is a function such that 1a(a) = a for all a 
A. Clearly, when the inverse of the function exists, then we have that f~'!of = 1, 
and f-of-! = 1p. 


Theorem 2.3 (Inverse of Function) 
A total function has an inverse if and only if it is bijective. 


Proof Suppose f : A — B has an inverse f-'. Then we show that f is bijective. 


We first show that f is one to one. 
Suppose f(x,) =f (x2) then 


fF) =f" F (2) 
=> fof (x1) =f of (x2) 
=> Ia(x1) = La(x2) 
> XxX, =X. 


Next, we first show that fis onto. Let. 
b € Band let a = f ‘(b) then 


f(a) =f(f~'(b)) = band sof is surjective. 


The second part of the proof is concerned with showing that if f : A — B is 
bijective, then it has an inverse f '. Clearly, since fis bijective we have that for each 
a € A there exists a unique b € B such that f (a) = b. 

Define g: B — A by letting g(b) be the unique a in A such that f(a) = b. Then 
we have that 


gof(a) = g(b) = aandf og(b) = f(a) =b. 


Therefore, g is the inverse of f 
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In this section, we discuss the applications of functions to functional programming, 
which is quite distinct from the imperative programming languages used in com- 
puting. Functional programming differs from imperative programming in that it 
involves the evaluation of mathematical functions, whereas imperative program- 
ming involves the execution of sequential (or iterative) commands that change the 
state. For example, the assignment statement alters the value of a variable, and the 
value of a given variable x may change during program execution. 

There are no changes of state for functional programs, and the fact that the value 
of x will always be the same makes it easier to reason about functional programs 
than imperative programs. Functional programming languages provide referential 
transparency: i.e. equals may be substituted for equals, and if two expressions have 
equal values, then one can be substituted for the other in any larger expression 
without affecting the result of the computation. 

Functional programming languages use higher order functions, '° recursion, lazy 
and eager evaluation, monads'! and Hindley—Milner type inference systems.'* 
These languages are mainly been used in academia, but there has been some 
industrial use, including the use of Erlang for concurrent applications in industry. 
Alonzo Church developed Lambda calculus in the 1930s, and it provides an 
abstract framework for describing mathematical functions and their evaluation. It 
provides the foundation for functional programming languages. Church employed 
lambda calculus to prove that there is no solution to the decision problem for 
first-order arithmetic in 1936 (discussed in Chap. 13). 

Lambda calculus uses transformation rules, and one of these rules is variable 
substitution. The original calculus developed by Church was untyped, but typed 
lambda calculi have since been developed. Any computable function can be 
expressed and evaluated using lambda calculus, but there is no general algorithm to 
determine whether two arbitrary lambda calculus expressions are equivalent. 
Lambda calculus influenced functional programming languages such as Lisp, ML 
and Haskell. 

Functional programming uses the notion of higher order functions. Higher order 
takes other functions as arguments, and may return functions as results. The 
derivative function “/,,f(x) =f’ (x) is a higher order function. It takes a function as 
an argument and returns a function as a result. For example, the derivative of the 
function Sin(x) is given by Cos(x). Higher order functions allow currying which is a 
technique developed by Schonfinkel. It allows a function with several arguments to 


‘Higher order functions are functions that take functions as arguments or return a function as a 
result. They are known as operators (or functionals) in mathematics, and one example is the 
derivative function Wy that takes a function as an argument and returns a function as a result. 
''Monads are used in functional programming to express input and output operations without 
introducing side effects. The Haskell functional programming language makes use of the uses this 
feature. 

This is the most common algorithm used to perform type inference. Type inference is concerned 
with determining the type of the value derived from the eventual evaluation of an expression. 
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be applied to each of its arguments one at a time, with each application returning a 
new (higher order) function that accepts the next argument. This allows a function 
of n-arguments to be treated as n applications of a function with 1-argument. 

John McCarthy developed LISP at MIT in the late 1950s, and this language 
includes many of the features found in modern functional programming lan- 
guages.’ Scheme built upon the ideas in LISP. Kenneth Iverson developed APL"* 
in the early 1960s, and this language influenced Backus’s FP programming lan- 
guage. Robin Milner designed the ML programming language in the early 1970s. 
David Turner developed Miranda in the mid-1980s. The Haskell programming 
language was released in the late 1980s. 


Miranda Functional Programming Language 


Miranda was developed by David Turner at the University of Kent in the mid-1980s 
[5]. It is a non-strict functional programming language: i.e. the arguments to a 
function are not evaluated until they are actually required within the function being 
called. This is also known as lazy evaluation, and one of its main advantages is that 
it allows infinite data structures to be passed as an argument to a function. Miranda 
is a pure functional language in that there are no side-effect features in the language. 
The language has been used for 


— Rapid prototyping, 
— Specification language and 


— Teaching Language. 


A Miranda program is a collection of equations that define various functions and 
data structures. It is a strongly typed language with a terse notation. 


z = sqr p/sqrq 


sqrk =k*k 
p=a+t+b 
q=a-—b 
a= 10 
b=5. 


The scope of a formal parameter (e.g. the parameter k above in the function sqr) 
is limited to the definition of the function in which it occurs. 

One of the most common data structures used in Miranda is the list. The empty 
list is denoted by [], and an example of a list of integers is given by [1, 3, 4, 8]. Lists 
may be appended to by using the ‘ ++ ’ operator. For example: 


‘Lisp is a multi-paradigm language rather than a functional programming language. 
‘Iverson received the Turing Award in 1979 for his contributions to programming language and 
mathematical notation. The title of his Turing award paper was “Notation as a tool of thought”. 
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(1,3,5}+ + [2,4] = [1,3,5,2, 4]. 
The length of a list is given by the ‘#’ operator: 
#(1,3] = 2. 


The infix operator ‘:’ is employed to prefix an element to the front of a list. For 
example: 


5: [2, 4, 6] is equal to [5, 2, 4, 6]. 
The subscript operator ‘!’ is employed for subscripting. For example: 
Nums = [5, 2, 4, 6] then Nums!0is5. 


The elements of a list are required to be of the same type. A sequence of 
elements that contains mixed types is called a tuple. A tuple is written as follows: 


Employee = (“Hotmes’ , 221B Baker St. London’, 50, Detective’) ; 


A tuple is similar to a record in Pascal, whereas lists are similar to arrays. Tuples 
cannot be subscripted but their elements may be extracted by pattern matching. 
Pattern matching is illustrated by the well-known example of the factorial function: 


facO = 1 
fac(n+ 1) = (n+ 1) * facn. 


The definition of the factorial function uses two equations, distinguished by the 
use of different patterns in the formal parameters. Another example of pattern 
matching is the reverse function on lists: 


reverse [] = [] 
reverse(a : x) = reverse x + + [a]. 


Miranda is a higher order language, and it allows functions to be passed as 
parameters and returned as results. Currying is allowed and this allows a function of 
n-arguments to be treated as n applications of a function with 1-argument. Function 
application is left-associative: i.e. f x y means (f x) y. That is, the result of applying 
the function f to x is a function, and this function is then applied to y. Every function 
with two or more arguments in Miranda is a higher order function. 
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2.6 Review Questions 


. What is a set? A relation? A function? 

. Explain the difference between a partial and a total function. 

. Explain the difference between a relation and a function. 

. Determine A x B where A = {a, b, c, d} and B = {1, 2, 3}. 

. Determine the symmetric difference AAB where A = {a, b, c, d} and 
B = {c, d, e}. 

6. What is the graph of the relation < on the set A = {2, 3, 4}. 

7. What is the composition of S and R (i.e. S o R), where R is a relation 
between A and B, and S is a relation between B and C. The sets A, B, C 
are defined as A = {a, b, c,d}, B = {e,f, g}, C = {h, i,j,k} andR = {(a, 
e), (b, &), (b, 8), (c, &), df} with S = {(e, h), (ce, b), Fis Fs (gh). 

8. What is the domain and range of the relation R where R = {(q, p), (a, r), 
(b, q)}. 

9. Determine the inverse relation R-' where R = {(a, 2), (a, 5), (b, 3), (b, 4), 
(c, 1}. 

10. Determine the inverse of the function f:RxR—R _ defined 
byf(x) =%52 (x #3) andy(3) =1 

11. Give examples of injective, surjective and bijective functions. 

12. Letn > 2 bea fixed integer. Consider the relation = defined by{(p, q): 


p.q € Z,n|(q-p)}: 


nA WN eR 


a. Show = is an equivalence relation. 
b. What are the equivalence classes of this relation? 


13. Describe the differences between imperative programming languages and 
functional programming languages. 


2.7. Summary 


This chapter provided an introduction to set theory, relations and functions. Sets are 
collections of well-defined objects; a relation between A and B indicates relation- 
ships between members of the sets A and B; functions are a special type of relation 
where there is at most one relationship for each element a € A with an element in 
B. 

A set is a collection of well-defined objects that contains no duplicates. There are 
many examples of sets such as the set of natural numbers N, the integer numbers Z 
and so on. 
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The Cartesian product allows a new set to be created from existing sets. The 
Cartesian product of two sets S and T (denoted by S' x 7) is the set of ordered pairs 
{(s, t)|s € S,t € Th. 

A binary relation R(A, B) is a subset of the Cartesian product (A x B) of A and 
B where A and B are sets. The domain of the relation is A and the codomain of the 
relation is B. The notation aRb signifies that there is a relation between a and b and 
that (a4, b)E€R. An n-ary relation R(A,,A2,...A,) is a subset of 
(Ay x Az X +++ X An). 

A total function f : A — B is a special relation such that for each element a € A, 
there is exactly one element b € B. This is written as f(a) = b. A function is a 
relation but not every relation is a function. 

The domain of the function (denoted by dom /f) is the set of values in A for which 
the function is defined. The domain of the function is A provided that fis a total 
function. The codomain of the function is B. 

Functional programming is quite distinct from imperative programming in that 
there is no change of state, and the value of the variable x remains the same during 
program execution. This makes functional programs easier to reason about than 
imperative programs. 
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Number Theory 


3.1. Introduction 


Number theory is the branch of mathematics that is concerned with the mathe- 
matical properties of the natural numbers and integers. These include properties 
such as the parity of a number; divisibility; additive and multiplicative properties; 
whether a number is prime or composite; the prime factors of a number; the greatest 
common divisor and least common multiple of two numbers; and so on. 

Number theory has many applications in computing including cryptography and 
coding theory. For example, the RSA public key cryptographic system relies on its 
security due to the infeasibility of the integer factorization problem for large 
numbers. 

There are several unsolved problems in number theory and especially in prime 
number theory. For example, Goldbach’s' conjecture states that every even integer 
greater than two is the sum of two primes, and this result has not been proved to 
date. Fermat’s~ Last Theorem (Fig. 3.1) states that there is no integer solution to 
x” + y" = z” for n > 2, and this result remained unproved for over 300 years until 
Andrew Wiles finally proved it in the mid-1990s. 


‘Goldbach was an eighteenth-century German mathematician and Goldbach’s conjecture has been 
verified to be true for all integers n < 12 * 10!” 

*Pierre de Fermat was the 17th French civil servant and amateur mathematician. He occasionally 
wrote to contemporary mathematicians announcing his latest theorem without providing the 
accompanying proof and inviting them to find the proof. The fact that he never revealed his proofs 
caused a lot of frustration among his contemporaries, and in his announcement of his famous last 
theorem he stated that he had a wonderful proof that was too large to include in the margin. He 
corresponded with Pascal, and they did some early work on the mathematical rules of games of 
chance and early probability theory. He also did some early work on the Calculus. 
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Fig. 3.1 Pierre de Fermat 
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Fig. 3.2 Pythagorean triples 


The natural numbers N consist of the numbers {1, 2, 3, ...}. The integer 
numbers Z consist of {... —2, -1, 0, 1, 2, ...}. The rational numbers Q consist of all 
the numbers of the form {”/, where p and q are integers and q # 0}. The real 
numbers R is defined to be the set of converging sequences of rational numbers, 
and they are a superset of the rational numbers. They contain the rational and 
irrational numbers. The complex numbers C consist of all the numbers of the form 
{a + bi where a, b € R and i = ¥-1}. 

Pythagorean triples (Fig. 3.2) are combinations of three whole numbers that 
satisfy Pythagoras’ equation x? + y” = 2’. There are an infinite number of such 
triples, and an example of such a triple is 3, 4, 5 since 37447 = 57, 

The Pythagoreans discovered the mathematical relationship between the har- 
mony of music and numbers, and their philosophy was that numbers are hidden in 
everything from music to science and nature. This led to their philosophy that 
“everything is number”. 
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A square number (Fig. 3.3) is an integer that is the square of another integer. For 
example, the number 4 is a square number since 4 = 2. Similarly, the number 9 and 
the number 16 are square numbers. A number n is a square number if and only if 
one can arrange the n points in a square. For example, the square numbers 4, 9 and 
16 are represented in squares as follows. 

The square of an odd number is odd whereas the square of an even number is 
even. This is clear since an even number is of the form n = 2k for some k, and so 
n° = 4K which is even. Similarly, an odd number is of the form n = 2k + 1 and so 
n> = 4k? + 4k + 1 which is odd. 

A rectangular number (Fig. 3.4) n may be represented by a vertical and hori- 
zontal rectangle of n points. For example, the number 6 may be represented by a 
rectangle with length 3 and breadth 2, or a rectangle with length 2 and breadth 3. 
Similarly, the number 12 can be represented by a 4 x 3 or a3 x 4 rectangle. 

A triangular number (Fig. 3.5) may be represented by an equilateral triangle of 
n points. It is the sum of & natural numbers from 1 to k. = That is, 


n= leIbone LE 


Parity of Integers 


The parity of an integer refers to whether the integer is odd or even. An integer 7 is 
odd if there is a remainder of one when it is divided by two, and it is of the form 
n = 2k + 1. Otherwise, the number is even and of the form n = 2k. 

The sum of two numbers is even if both are even or both are odd. The product of 
two numbers is even if at least one of the numbers is even. These properties are 
expressed as 


even + even = even 


even + odd = odd 


odd + odd = even 


Fig. 3.3 Square numbers 
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Fig. 3.5 Triangular numbers 


even X even = even 
even x odd = even 


odd x odd = odd. 


Divisors 


Let a and b be integers with a 4 0 then a is said to be a divisor of b (denoted by a | 
b) if there exists an integer k such that b = ka. 

A divisor of 7 is called a trivial divisor if it is either 1 or n itself; otherwise, it is 
called a non-trivial divisor. A proper divisor of nis a divisor of n other than n itself. 


DEFINITION (PRIME NUMBER) 


A prime number is a number whose only divisors are trivial. There are an infinite 
number of prime numbers. 

The fundamental theorem of arithmetic states that every integer number can be 
factored as the product of prime numbers. 


Mersenne Primes 


Mersenne primes are prime numbers of the form 2” — 1 where p is a prime. They are 
named after Marin Mersenne (Fig. 3.6) who was the 17th French monk, philosopher 
and mathematician. Mersenne did some early work in identifying primes of this 
format, and there are 47 known Mersenne primes. It remains an open question as to 
whether there are an infinite number of Mersenne primes. 
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Fig. 3.6 Marin Mersenne 


Properties of Divisors 


(i) a|banda|cthena|b+c 
(ii) a|bthena | be 
(iii) a|bandb|cthena|c 


Proof (of i) Suppose a | b and a|c then b = kya and c = kya. 
Then b+c =kya+koa = (kj +ky)a and soa|bt+c. 


Proof (of iii) Suppose a | b and b | c then b = kya and c = kab. 
Then c = kyb = (kok)a and thus a | c. 


Perfect and Amicable Numbers 


Perfect and amicable numbers have been studied for millennia. A positive integer 
m is said to be perfect if it is the sum of its proper divisors. Two positive integers 
m and n are said to be an amicable pair if m is equal to the sum of the proper 
divisors of n and vice versa. 

A perfect number is a number whose divisors add up to the number itself. For 
example, the number 6 is perfect since it has divisors 1, 2,3 and 1+2+3=6. 

Perfect numbers are quite rare and Euclid showed that 2” i (2? — 1) is an even 
perfect number whenever (2” — 1) is prime. Euler later showed that all even perfect 
numbers are of this form. It is an open question as to whether there are any odd 
perfect numbers, and if such an odd perfect number N was to exist then N > 10'°°°, 

A prime number of the form (2? — 1) where p is prime is called a Mersenne 
prime. Mersenne primes are quite rare and each Mersenne prime generates an even 
perfect number and vice versa. That is, there is a one-to-one correspondence 
between the number of Mersenne primes and the number of even perfect numbers. 
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It remains an open question as to whether there are an infinite number of 
Mersenne primes and perfect numbers. 

An amicable pair of numbers is a pair of numbers such that each number is the sum 
of divisors of the other number. For example, the numbers 220 and 284 are an 
amicable pair since the divisors of 220 are 1,2, 4,5, 10, 11, 20, 22, 44, 55, 110, which 
have sum 284, and the divisors of 284 are 1, 2, 4, 71, 142, which have sum 220. 


Theorem 3.1 (Division Algorithm) For any integer a and any positive integer b, 
there exists unique integers q and r such that 


a=bq+r O<r<b. 


Proof The first part of the proof is to show the existence of integers g and r such 
that the equality holds, and the second part of the proof is to prove the uniqueness 
of q and r. 

Consider .... —3b, —2b, —b, 0, b, 2b, 3b, ... then there must be an integer g such 
that 


qb <a<(q+I)b. 


Then a - qb =rwithO < r<band soa = bg + +r, and the existence of g and 
r is proved. 

The second part of the proof is to show the uniqueness of g and r. Suppose q, 
and r, also satisfy a=bq,+r, with 0 < r,; <b, and suppose r<r,. Then 
bq+r=bqi +n and so b(q — qi) =r —r and clearly 0<(r,; — r)<b. There- 
fore, b | (r; — r) which is impossible unless 7, — r = 0. Hence, r = r; and qg = q. 


Theorem 3.2 (Irrationality of Square Root of Two) The square root of two is an 
irrational number (i.e. it cannot be expressed as the quotient of two integer 
numbers). 


Proof The Pythagoreans* discovered this result, and it led to a crisis in their 
community as number was considered to be the essence of everything in their 
world. The proof is indirect: i.e. the opposite of the desired result is assumed to be 


Pythagoras of Samos (a Greek island in the Aegean sea) was an influential ancient mathematician 
and philosopher of the sixth century B.C. He gained his mathematical knowledge from his travels 
throughout the ancient world (especially in Egypt and Babylon). He became convinced that 
everything is number, and he and his followers discovered the relationship between mathematics 
and the physical world as well as relationships between numbers and music. On his return to 
Samos, he founded a school and he later moved to Croton in Southern Italy to set up a school. This 
school and the Pythagorean brotherhood became a secret society with religious beliefs such as 
reincarnation, and they were focused on the study of mathematics. They maintained secrecy of the 
mathematical results that they discovered. Pythagoras is remembered today for Pythagoras’ 
theorem, which states that for a right-angled triangle the square of the hypotenuse is equal to the 
sum of the squares of the other two sides. The Pythagoreans discovered the irrationality of the 
square root of two and as this result conflicted in a fundamental way with their philosophy that 
number is everything, they suppressed the truth of this mathematical result. 
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correct, and it is showed that this assumption leads to a contradiction. Therefore, the 
assumption must be incorrect and so the result is proved. 

Suppose 72 is rational then it can be put in the form ’/, where p and q are 
integers and q # 0. Therefore, we can choose p, q to be co-prime (i.e. without any 
common factors) and so 


("/.)° = 2 
pig=2 
p’=2q¢ 
2|p 
2|p 
p= 2k 
p= 4k 
4k=2q¢° 
2 k= q 
2|q 
2|q 


This is a contradiction as we have chosen p and gq to be co-prime, and our 
assumption that there is a rational number that is the square root of two results in a 
contradiction. Therefore, this assumption must be false, and we conclude that there 
is no rational number whose square is two. 
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3.3 Prime Number Theory 


A positive integer n > 1 is called prime if its only divisors are n and 1. A number 
that is not a prime is called composite. 


Properties of Prime Numbers 


(i) There are an infinite number of primes. 
(ii) There is a prime number p between n and n! + 1 such that n<p<n!+1. 
(iii) If n is composite, then n has a prime divisor p such that p < yn. 
(iv) There are arbitrarily large gaps in the series of primes (given any k > 0, there 
exist k consecutive composite integers). 


Proof (i) Suppose there are a finite number of primes and they are listed as pj, p2, 
P3, ---» Py Then consider the number N obtained by multiplying all known primes 
and adding one. That is, 


N = pip2p3-- Pet 1. 
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Clearly, N is not divisible by any of p), po, p3, ..., Dx Since they all leave a 
remainder of 1. Therefore, N is either a new prime or divisible by a prime gq (that is 
not in the list of pi, po, p3, ---, Dx): 

This is a contradiction since this was the list of all the prime numbers, and so the 
assumption that there are a finite number of primes is false, and we deduce that 
there are an infinite number of primes. 


Proof (ii) Consider the integer N=n! +1. If N is prime then we take 
p =N. Otherwise, N is composite and has a prime factor p. We will show that 
p>n. 

Suppose p < n then p|n! and since p | N we have p | n! + 1 and therefore p | 1, 
which is impossible. Therefore, p > and the result is proved. 


Proof (iii) Let p be the smallest prime divisor of n. Since n is composite n = wv, 
and clearly p < u and p < v. Then p < uv=nandsop < vn. 


Proof (iv) Consider the k consecutive integers (k+1)!+2, (kK+1)!+3,..., 
(k+1)!+k, (k+1)!+k+1. Then each of these is composite since j|(k+ 1)!+] 
where 2<j<k+1. 


3.3.1 Algorithms 


An algorithm is a well-defined procedure for solving a problem, and it consists of a 
sequence of steps that takes a set of values as input, and produces a set of values as 
output. It is an exact specification of how to solve the problem, and it explicitly 
defines the procedure so that a computer program may implement the algorithm. 
The origin of the word ‘algorithm’ is from the name of the 9th Persian mathe- 
matician, Muhammad al-Khwarizmi. 

It is essential that the algorithm is correct and that it terminates in a reasonable 
time. This may require mathematical analysis of the algorithm to demonstrate its 
correctness and efficiency, and to show that termination is within an acceptable time 
frame. There may be several algorithms to solve a problem, and so the choice of the 
best algorithm (e.g. fastest/most efficient) needs to be considered. For example, 
there are several well-known sorting algorithms (e.g. merge sort and insertion sort), 
and the merge sort algorithm is more efficient [o(7 lg )] than the insertion sort 
algorithm [0(n*)]. 

An algorithm may be implemented by a computer program written in some 
programming language (e.g. C++ or Java). The speed of the program depends on 
the algorithm employed, the input value(s), how the algorithm has been imple- 
mented in the programming language, the compiler, the operating system and the 
computer hardware. 
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An algorithm may be described in natural language (care is needed to avoid 
ambiguity), but it is more common to use a more precise formalism for its 
description. These include pseudo code (an informal high-level language descrip- 
tion); flowcharts; a programming language such as C or Java; or a formal speci- 
fication language such as VDM or Z. We shall mainly use a natural language and 
pseudo code to describe an algorithm. Among the early algorithms developed was 
an algorithm to determine the prime numbers up to a given number n, and Euclid’s 
algorithm for determining the greatest common divisor of two natural numbers. 
These are discussed below. 


Algorithm for Determining Primes 


The Sieve of Eratosthenes algorithm (Fig. 3.7) is a famous algorithm for deter- 
mining the prime numbers up to a given number n. It was developed by the 
Hellenistic mathematician, Eratosthenes. 

The algorithm involves first listing all of the numbers from 2 to n. The first step 
is to remove all the multiples of two up to n; the second step is to remove all the 
multiples of three up to n; and so on. 

The kth step involves removing multiples of the kth prime p; up to n, and the 
steps in the algorithm continue while p < /n. The numbers remaining in the list are 
the prime numbers from 2 to n. 


1. List the integers from 2 to n. 
2. For each prime p, up to vn, remove all the multiples of p,. 
3. The numbers remaining are the prime numbers between 2 and n. 


The list of primes between | and 50 is given in Fig. 3.7. They are 2, 3, 5, 7, 11, 
13, 17, 19, 23, 29, 31, 37, 41, 43 and 47. 
Theorem 3.3 (Fundamental Theorem of Arithmetic) Every natural number n > 1 
may be written uniquely as the product of primes: 
n= py'prp3.. pe. 


Proof There are two parts to the proof. The first part shows that there is a fac- 
torization, and the second part shows that the factorization is unique. 


Fig. 3.7 Primes between | and 50 
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Part(a) 

If n is prime, then it is a product with a single prime factor. Otherwise, n can be 
factored into the product of two numbers ab where a > | and b > 1. The argument 
can then be applied to each of a and b each of which is either prime or can be 
factored as the product of two numbers both of which are greater than one. Con- 
tinue in this way with the numbers involved decreasing with every step in the 
process until eventually all of the numbers are prime. (This argument can be made 
more rigorous using strong induction.) 


Part(b) 
Suppose the factorization is not unique, and let n > 1 be the smallest number that 
has more than one factorization of primes. Then n may be expressed as follows: 


nN = Pip2p3---Pk = G19243-- -Gr- 


Clearly, k>1 and r>1 and pjz~ q; for G@=1, ..., k) and G=1, ..., 7) as 
otherwise we could construct a number smaller than n (e.g. n/p; where p; = q;) that 


has two distinct factorizations. Next, without loss of generality take p, < q, and 
define the number N by 


N= (1 — P1)9293- Or 
= P1P2P3---Pk — P19293---4r 
= Pi (P2P3- « Pk — 243- - Gr): 


Clearly, | <N<n and so N is uniquely factorizable into primes. However, 
clearly p; is not a divisor of (g, — p;), and so N has two distinct factorizations, 
which is a contradiction of the choice of n. 


3.3.2. Greatest Common Divisors (GCD) 


Let a and b be integers, not both zero. The greatest common divisor d of a and b is a 
divisor of a and b (i.e. d| a and d | b), and it is the largest such divisor (ie. if k | 
a and k | b then k | d). It is denoted by gcd (a, b). 


Properties of Greatest Common Divisors 
Gj) Let a and b be integers not both zero, then there exist integers x and y such that 
d = gcd (a,b) = ax+by. 


(ii) Let a and b be integers not both zero, then the set S = {ax + by where x, 
y € Z} is the set of all the multiples of d = gcd (a, b). 
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Proof (of i) Consider the set of all the linear combinations of a and b forming the 
set {ka + nb: k, n € Z}. Clearly, this set includes positive and negative numbers. 
Choose x and y such that m = ax + by is the smallest positive integer in the set. 
Then we shall show that m is the greatest common divisor. 

We know from the division algorithm that a = mg + r where 0 < r<m. Thus 


r = a—mgq = a—(ax+ by)q = (1 — qx)a+ (—yq)b. 


ris a linear combination of a and b and so r must be O from the definition of m. 
Therefore, m | aand similarly m |b and so mis acommon divisor of a and b. Since, the 
greatest common divisor dis such that d| a and d | bandd < m, wemust haved = m. 


Proof (of ii) This follows since d| a and d|b = > d|ax + by for all integers x and 
y and so every element in the set S = {ax + by where x, y € Z} is a multiple of d. 


Relatively Prime 

Two integers a, b are relatively prime if gcd(a, b) = 1. 
Properties 

If p is a prime and p | ab then p | a or p| b. 


Proof Suppose p | a then from the results on the greatest common divisor, we have 
gcd (a, p) = 1. That is, 


ra+sp=1 
» rab+spb=b 
» p|b (since p|rab and p | spb and so p| rab + spb) 


3.3.3. Least Common Multiple (LCM) 


If m is a multiple of a and m is a multiple of b, then it is said to be a common 
multiple of a and b. The least common multiple is the smallest of the common 
multiples of a and b and it is denoted by Icm (a, b). 


Properties 


If x is acommon multiple of a and b, then m | x. That is, every common multiple of 
a and b is a multiple of the least common multiple m. 


Proof We assume that both a and b are non-zero as otherwise the result is trivial 
(since all the common multiples are 0). Clearly, by the division algorithm, we have 


x = mq-+r where O<r<m. 
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Since x is a common multiple of a and b, we have a | x and b | x and also that 
a| mand b | m. Therefore, a | r and b | r, and so r is acommon multiple of a and b 
and since m is the least common multiple we have r is 0. Therefore, x is a multiple 
of the least common multiple m as required. 


3.3.4 Euclid’s Algorithm 


Euclid’s* algorithm is one of the oldest known algorithms, and it provides a pro- 
cedure for finding the greatest common divisor of two numbers. It appears in 
Book VII of Euclid’s Elements, and the algorithm was known prior to Euclid 
(Fig. 3.8). 


Lemma 


Let a, b, q and r be integers with b > 0 and0 < r <b such that a = bq + r. Then 
gcd(a, b) = gcd(b, 1). 


Proof Let K = gcd(a, b) and let L = gced(b, r), and we therefore need to show that 
K = L. Suppose m is a divisor of a and b, then as a = bq + r we have m is a divisor 
of r and so any common divisor of a and 5 is a divisor of r. 

Similarly, any common divisor n of b and r is a divisor of a. Therefore, 
the greatest common divisor of a and b is equal to the greatest common divisor 
of b and r. 


Theorem 3.4 (Euclid’s Algorithm) Euclid’s algorithm for finding the greatest 
common divisor of two positive integers a and b involves applying the division 
algorithm repeatedly as follows: 


a=bqt+n 0<r,<b 
b=ngt+n 0<n<rn 
rT) = hgotrs 0<73<%m 


Tn—-2 = Vn-19n-1 + Tn O0<tn<tn-1 
Tn-1 = Tn n- 


Then 7, (i.e. the last non-zero remainder) is the greatest common divisor of a and 
b: i.e. gcd(a,b) = rp. 


“Euclid was a third-century B.C. Hellenistic mathematician and is considered the father of 
geometry. 
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Fig. 3.8 Euclid of 
Alexandria 


Proof It is clear from the construction that r,, is a divisor of 7,1, Tp_2, ---) 13, 12. 11 
and of a and b. Clearly, any common divisor of a and b will also divide r,. Using 
the results from the lemma above, we have 


gcd(a, b) 


= gcd(b, r1) 
= ged(rir2) 


= gcd(Tn-27n-1) 
= gcd(Tn-1, Tn) 
=p. 
Lemma 
Let n be a positive integer greater than one, then the positive divisors of n are 


precisely those integers of the form: 


d= pipe ph. ; phk (where 0 < ; < aj) 


where the unique factorization of n is given by 


k 


n= py pr” ps... prs. 
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Proof Suppose d is a divisor of n, then n = dq. By the unique factorization theo- 
rem, the prime factorization of n is unique, and so the prime numbers in the 
factorization of d must appear in the prime factors py, po, p3, ..., Pe Of n. 

Clearly, the power f; of p; must be less than or equal to ;: i.e. B; <a. Con- 
versely, whenever [; < a; then clearly d divides n. 


3.3.5 Distribution of Primes 


We already have shown that there are an infinite number of primes. However, most 
integer numbers are composite and a reasonable question to ask is how many primes 
are there less than a certain number. The number of primes less than or equal to x is 
known as the prime distribution function (denoted by m(x)) and it is defined by 


m(x) = & 1 (where pis prime). 


p<Tx 


The prime distribution function satisfies the following properties: 


The first property expresses the fact that most integer numbers are composite, 
and the second property expresses the fact that there are an infinite number of prime 
numbers. 

There is an approximation of the prime distribution function in terms of the 
logarithmic function (“/n x) as follows: 


n(x) 


xsoo x/ Inx 


= 1 (Prime Number Theorem). 


The approximation x/In x to (x) gives an easy way to determine the approxi- 
mate value of m(x) for a given value of x. This result is known as the Prime Number 
Theorem, and Gauss originally conjectured this theorem. 


Palindromic Primes 


A palindromic prime is a prime number that is also a palindrome (i.e. it reads the 
same left to right as right to left). For example, 11, 101, 353 are all the palindromic 
primes. 
All the palindromic primes (apart from 11) have an odd number of digits. It is an 
open question as to whether there are an infinite number of palindromic primes. 
Let o(m) denote the sum of all the positive divisors of m (including m): 


o(m) => Ldlm d. 
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Let s(m) denote the sum of all the positive divisors of m (excluding m): 


Clearly, s(m) = m and o(m) = 2m when m is a perfect number. 


Theorem 3.5 (Euclid—Euler Theorem) The positive integer n is an even perfect 
number if and only if n= 2?~'(2?—1) where 2? — 1 is a Mersenne prime. 


Proof Suppose n = 2?~'(2?—1) where 2” — 1 is a Mersenne prime, then 


Therefore, n is a perfect number since o(n) = 2n. 

The next part of the proof is to show that any even perfect number must be of the 
form above. Let n be an arbitrary even perfect number (n = 2?~'g) with g odd and 
so the gcd (2?"', q) = 1, and so 


= 2n (since nis perfect) 
= 2.2P"1q 
— 2 q 


Therefore, 
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Therefore, (2?—1)s(q) = q. 

Therefore, d = s(q) is a proper divisor of g. However, s(q) is the sum of all the 
proper divisors of g including d, and so d is the only proper divisor of q and d= 1. 
Therefore, g = (2” — 1) is a Mersenne prime. 


Euler Function 


The Euler’ @ function (also known as the totient function) is defined for a given 
positive integer n to be the number of positive integers k less than n that are 
relatively prime to n. Two integers a, b are relatively prime if gcd(a,b) = 1 
(Fig. 3.9). 


o(n)= LX 1. where gcd(k,n) = 1. 


3.4 Theory of Congruences® 


Let a be an integer and n a positive integer greater than |, then (a mod n) is defined 
to be the remainder 7 when a is divided by n. That is, 


a=kn+r where 0<r<n. 


Definition 


Suppose a, b are integers and n a positive integer, then a is said to be congruent to 
b modulo n denoted by a = b (mod n) if they both have the same remainder when 
divided by n. 

This is equivalent to n being a divisor of (a — b) or n | (a — b) since we have 
a=kn+r and b=kyn+r, and so (a — b) = (ky — ky) n and son | (a — b). 


Theorem 3.6 Congruence modulo n is an equivalence relation on the set of 
integers: i.e. it is a reflexive, symmetric and transitive relation. 


Euler was an eighteenth-century Swiss mathematician who made important contributions to 
mathematics and physics. His contributions include graph theory (e.g. the well-known formula V — 
E + F = 2), calculus, infinite series, the exponential function for complex numbers and the totient 
function. 

°The theory of congruences was introduced by the German mathematician, Carl Friedrich Gauss. 
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Fig. 3.9 Leonard Euler 


Proof 
(i) Reflexive 
For any integer a, it is clear that a = a (mod n) since a—a=0.n 
(ii) Symmetric 
Suppose a = b (mod n) then a — b = kn. Clearly, b — a = -kn and so b= 
a (mod n). 
(iii) Transitive 
Suppose a = b(modn) and b = c(modn) 
=> a—b = kinand b—c = knn 
=> a—c = (a—b) + (b-c) 
=kin+kon 
= (ki + ko)n 
=> a=c(modn). 


Therefore, congruence modulo n is an equivalence relation, and an equivalence 
relation partitions a set S into equivalence classes (Theorem 2.2). The integers are 
partitioned into n equivalence classes for the congruence modulo n equivalence 
relation, and these are called congruence classes or residue classes. 
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The residue class of a modulo n is denoted by [a], or just [a] when 7 is clear. It 
is the set of all those integers that are congruent to a modulo n: 


[a],= {x: x € Zandx = a(modn)} = {a+kn:k € Z}. 


Any two equivalence classes [a] and [b] are either equal or disjoint: i.e. we have 
[a] = [b] or [a] N [b] = @. The set of all the residue classes modulo n is denoted by 


Z/nZ = Z, = {{a],: 0S a<n—1} = {(0),, [1], ---s [2 — U,}. 


For example, consider Z, the residue classes mod 4, then 


(O]4= {...,-8, -4,0,4,8,...... } 
(ee cere ee on A: } 
[2]4= {..., -6, -2,2,6, 10,...... } 
Bla= {-.-,-5,-1,3,7, 11,.. 0. 1, 


The reduced residue class is a set of integers r; such that (7;, 2) = 1 and r; is not 
congruent to 7; (mod n) for i 4 j, such that every x relatively prime to n is congruent 
modulo n i for some element 7; of the set. There are @(n) elements 
{r1,12,+++51o(n) } in the reduced residue class set S. 


Modular Arithmetic 


Addition, subtraction and multiplication may be defined in Z/nZ and are similar to 
these operations in Z. Given a positive integer n and integers a, b, c, d such that a= 
b (mod n) and c = d (mod n), then the following are properties of modular 
arithmetic. 


(i) a+c=b+d(modn) and a—c = b— d(modn). 


(ii) ac = bd(modn). 
(iii) a” = b"(modn)Vm €. 


Proof (of ii) Let a= kn+b and c = In+d for some k, | € Z, then 


= (kn+b)(In+d) 
= (kn)(In) + (kn)d + b(In) + bd 
= (knl + kd + bl)n+ bd 
= sn+bd (where s = knl + kd + bl) 


and so ac = bd(modn). 
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The three properties above may be expressed in the following equivalent 
formulation: 


(i) [a+c], =[b+d], and [a—c],= [b—d,. 
(ii) [ac], = [bd),,. 
(ii) [a",=[b"|,  Vme. 


Two integers x, y are said to be multiplicative inverses of each other modulo n if 
xy = 1(modn). 


However, x does not always have an inverse modulo n, and this is clear since, for 
example, [3]6 is a zero divisor modulo 6, i.e. [3]6. [2]¢ = [0] and so it does not 
have a multiplicative inverse. However, if n and x are relatively prime, then it is 
easy to see that x has an inverse (mod 7) since we know that there are integers k,/ 
such that kx + Jn = 1. 

Given n > 0 there are @(n) numbers b that are relatively prime to n, and so there 
are ~(n) numbers that have an inverse modulo n. Therefore, for p prime there are 
p-1 elements that have an inverse (mod p). 


Theorem 3.7 (Euler’s Theorem) Let a and n be positive integers with gcd(a,n) = 1. 
Then 


a®”) = 1(modn). 


Proof Let {r1,72,...,To(n)} be the reduced residue system (mod n). Then 
{ar|,Ar2,...,A%g(n)} is also a reduced residue system (mod n) since ar; = 
ar;(modn) and (a, n) = 1 implies that r; = r;(modn). 

For each r; there is exactly one r; such that ar; = r; (mod n), and different r; will 
have different corresponding ar;. Therefore, {ar;,arz,...,@g(n)} are just the 
residues module n of {r,,7r2,.. el gti but not necessarily in the same order. 
Multiplying, we get 


II (ar;) = II r; (modn) 
j=l i=! 
on) o(n) 
a?) II (7) = II r; (modn). 
j=l is! 


Since (r;, 2) = 1, we can deduce that a®”) = 1(modn) from the result that 
ax = ay(modn) and (a, n) = 1 then x = y(modn). 
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Theorem 3.8 (Fermat’s Little Theorem) Let a be a positive integer and p a prime. 
If gcd(a, p) = 1, then 


a?-' = 1(mod p). 


Proof This result is an immediate corollary to Euler’s theorem as o((p) = p—1. 
Theorem 3.9 (Wilson’s Theorem) /f p is a prime, then (p — 1)! = —1(modp). 

Proof Each element a€1,2,...»—1 has an inverse a’ such that 
aa~' = 1(modp). Exactly two of these elements 1 and p — | are their own inverse 
(i.e. x* = 1(modp) having two solutions 1 and p — 1). Therefore, the product 


1.2......» — 1(mod p) = p — 1(modp) = —1(modp). 


Diophantine equations 


The word ‘Diophantine’ is derived from the name of the third-century mathe- 
matician, Diophantus, who lived in the city of Alexandria in Egypt. Diophantus 
studied various polynomial equations of the form f(x, y, z, ...) = 0 with integer 
coefficients to determine which of them had integer solutions. 

A Diophantine equation may have no solution, a finite number of solutions or an 
infinite number of solutions. The integral solutions of a Diophantine equation f(x,y) 
= 0 may be interpreted geometrically as the points on the curve with integral 
coordinates. 


Example 


A linear Diophantine equation ax+by=c is an algebraic equation with two 
variables x and y, and the problem is to find integer solutions for x and y. 


3.5 Binary System and Computer Representation 
of Numbers 


Arithmetic has traditionally been done using the decimal notation,’ and this posi- 
tional number system involves using the digits 0, 1, 2, ... 9. Leibniz® was one of the 
earliest people to recognize the potential of the binary number system, and this base 
2 system uses just two digits namely ‘0’ and ‘1’. Leibniz described the binary 
system in Explication de l’Arithmétique Binaire [1], which was published in 1703. 


7Other bases have been employed such as the segadecimal (or base-60) system employed by the 
Babylonians. The decimal system was developed by Indian and Arabic mathematicians during 
800-900AD, and it was introduced to Europe in the late twelfth/early thirteenth century. It is 
known as the Hindu—Arabic system. 

SWilhelm Gottfried Leibniz was a German philosopher, mathematician and inventor in the field of 
mechanical calculators. He developed the binary number system used in digital computers, and 
invented the Calculus independently of Sir Isaac Newton. He was embroiled in a bitter dispute 
towards the end of his life with Newton, as to who developed the calculus first. 
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Table 3.1 Binary number system 


Binary Dec Binary Dec Binary Dec Binary Dec 
0000 0 0100 4 1000 8 1100 12 
0001 1 0101 5 1001 8) 1101 13 
0010 2 0110 6 1010 10 1110 14 
0011 3 0111 7 1011 11 1111 15 


His 1703 paper describes how binary numbers may be added, subtracted, multiplied 
and divided, and Leibniz was an advocate of their use. 

The number two is represented by 10, the number four by 100 and so on. A table 
of values for the first 15 binary numbers is given in Table 3.1. 

The binary number system (base 2) is a positional number system, which uses 
two binary digits 0 and 1, and an example binary number is 1001.01, which 
represents 1 x 27+1+1x 27>? =84140.25 = 9.25. 

The binary system is ideally suited to the digital world of computers, as a binary 
digit may be implemented by an on-off switch. In the digital world, devices that 
store information or data on permanent storage media such as discs, and CDs, or 
temporary storage media such as random access memory (RAM) consist of a large 
number of memory elements that may be in one of two states (i.e. on or off). 

The digit 1 represents that the switch is on, and the digit 0 represents that the 
switch is off. Claude Shannon showed in his Master’s thesis [2] that the binary 
digits (i.e. 0 and 1) can be represented by electrical switches. This allows binary 
arithmetic and more complex mathematical operations to be performed by relay 
circuits, provided the foundation of digital computing. 

The decimal system (base 10) is more familiar for everyday use, and there are 
algorithms to convert numbers from decimal to binary and vice versa. For example, 
to convert the decimal number 25 to its binary representation, we proceed as 
follows: 


2 25 
| 12 
| 6 
|3 
|1 
|0 


—- eS OO kK 


The base 2 is written on the left and the number to be converted to binary is 
placed in the first column. At each stage in the conversion, the number in the first 
column is divided by 2 to form the quotient and remainder, which are then placed 
on the next row. For the first step, the quotient when 25 is divided by 2 is 12 and the 
remainder is 1. The process continues until the quotient is 0, and the binary rep- 
resentation result is then obtained by reading the second column from the bottom 
up. Thus, we see that the binary representation of 25 is 110013. 
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Similarly, there are algorithms to convert decimal fractions to binary represen- 
tation (to a defined number of binary digits as the representation may not terminate), 
and the conversion of a number that contains an integer part and a fractional part 
involves converting each part separately and then combining them. 

The octal (base 8) and hexadecimal (base 16) are often used in computing, as the 
bases 2, 8 and 16 are related bases and it is easy to convert between, as converting 
between binary and octal involves grouping the bits into groups of three on either 
side of the point. Each set of 3 bits corresponds to one digit in the octal repre- 
sentation. Similarly, the conversion between binary and hexadecimal involves 
grouping into sets of 4 digits on either side of the point. The conversion of the other 
way from octal to binary or hexadecimal to binary is equally simple, and involves 
replacing the octal (or hexadecimal) digit with the 3-bit (or 4-bit) binary 
representation. 

Numbers are represented in a digital computer as sequences of bits of fixed 
length (e.g. 16-bits and 32-bits). There is a difference in the way in which integers 
and real numbers are represented, with the representation of real numbers being 
more complicated. 

An integer number is represented by a sequence (usually 2 or 4) bytes where 
each byte is 8 bits. For example, a 2-byte integer has 16 bits with the first bit used as 
the sign bit (the sign is | for negative numbers and 0 for positive integers), and the 
remaining 15 bits represent the number. This means that two bytes may be used to 
represent all the integer numbers between —32768 and 32767. A positive number is 
represented by the normal binary representation discussed earlier, whereas a neg- 
ative number is represented using 2’s complement of the original number (i.e. 
0 changes to 1 and 1 changes to 0, and the sign bit is 1). All of the standard 
arithmetic operations may then be carried out using modulo 2 arithmetic. 

The representation of floating-point real numbers is more complicated, and a real 
number is represented to a fixed number of significant digits (the significand) and 
scaled using an exponent in some base (usually 2). That is, the number is repre- 
sented (approximated as) 


significand x base**P°""™", 


The significand (also called mantissa) and exponent have a sign bit. For 
example, in simple floating-point representation (4 bytes), the mantissa is generally 
24 bits and the exponent 8 bits, whereas for double precision (8 bytes) the mantissa 
is generally 53 bits and the exponent 11 bits. There is an IEEE standard for 
floating-point numbers (IEEE 754). 
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3.6 Review Questions 


1. Show that 


(i) if a|b then a | be. 
(ii) Ifa] b and c|d then ac | bd. 


2. Show that 1184 and 1210 are an amicable pair. 
3. Use the Euclidean algorithm to find g = gcd (5, c) where b = 42,823 and 
c = 6409, and find integers x and y such that bx+ cy = g. 


4. List all integers x in the range 1 <x < 100 such that x = 7 (mod 17). 
5. Evaluate @(m) for m= 1,2 3, ... 12. 
6. Determine a complete and reduced residue system modulo 12. 
7. Convert 767 to binary, octal and hexadecimal. 
8. Convert (you may need to investigate) 0.329 to binary (to 5 places). 
9. Explain the difference between binary, octal and hexadecimal. 
10. Find the 16-bit integer representation of —4961. 


3.7. Summary 


Number theory is concerned with the mathematical properties of natural numbers 
and integers. These include properties such as whether a number is prime or 
composite, the prime factors of a number, the greatest common divisor and least 
common multiple of two numbers and so on. 

The natural numbers N consist of the numbers {1, 2, 3, ... }. The integer 
numbers Z consist of {... —2,-1, 0, 1, 2, ...}. The rational numbers Q consist of all 
the numbers of the form {”/, where p and q are integers and g # 0}. Number theory 
has been applied to cryptography in the computing field. 

Prime numbers have no factors apart from themselves and one, and there are an 
infinite number of primes. The Sieve of Eratosthenes’ algorithm may be employed 
to determine prime numbers, and the approximation to the distribution of prime 
numbers less than a number n is given by the prime distribution function 
m(n) = n/Inn. Prime numbers are the key building blocks in number theory, and 
the fundamental theorem of arithmetic states that every number may be written 
uniquely as the product of factors of prime numbers. 

Mersenne primes and perfect numbers were considered, and it was shown that 
there is a one-to-one correspondence between the Mersenne primes and the even 
perfect numbers. 

Modulo arithmetic including addition, subtraction and multiplication were 
defined, and the residue classes and reduced residue classes were discussed. There 
are unsolved problems in number theory such as Goldbach’s conjecture that states 
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that every even integer is the sum of two primes. Other open questions include 
whether there are an infinite number of Mersenne primes and palindromic primes. 

We discussed the binary number system, which is ideally suited for digital 
computers. We discussed the conversion between binary and decimal systems, as 
well as the octal and hexadecimal systems. Finally, we discussed the representation 
of integers and real numbers on a computer. For more detailed information on 
number theory, see [3]. 
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Mathematical Induction and Recursion 


4.1 Introduction 


Mathematical induction is an important proof technique used in mathematics, and it 
is often used to establish the truth of a statement for all the natural numbers. There 
are two parts to a proof by induction, and these are the base step and the inductive 
step. The first step is termed the base case, and it involves showing that the 
statement is true for some natural number (usually the number 1). The second step 
is termed the inductive step, and it involves showing that if the statement is true for 
some natural number n = k, then the statement is true for its successor n = k + 1. 
This is often written as P(k) — P(k+1). 

The statement P(x) that is assumed to be true when n = k is termed the inductive 
hypothesis. From the base step and the inductive step, we infer that the statement is 
true for all the natural numbers (that are greater than or equal to the number 
specified in the base case). Formally, the proof technique used in mathematical 
induction is of the form!: 


(P(1) AVk(P(k) > P(k+1))) > Vn P(n). 


Mathematical induction (weak induction) may be used to prove a wide variety of 
theorems, and especially theorems of the form Vn P(n). It may be used to provide 
proof of theorems about summation formulae, inequalities, set theory, and the 
correctness of algorithms and computer programs. One of the earliest inductive 


proofs was the sixteenth-century proof that the sum of the first n odd integers is n”, 


'This definition of mathematical induction covers the base case of n = 1, and would need to be 
adjusted if the number specified in the base case is higher. 
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which was proved by Francesco Maurolico in 1575. Later mathematicians made the 
method of mathematical induction more precise. 

We distinguish between strong induction and weak induction, where strong 
induction also has a base case and an inductive step, but the inductive step is a little 
different. It involves showing that if the statement is true for all the natural numbers 
less than or equal to an arbitrary number k, then the statement is true for its 
successor k + 1. Structural induction is another form of induction and this math- 
ematical technique is used to prove properties about recursively defined sets and 
structures. 

Recursion is often used in mathematics to define functions, sequences and sets. 
However, care is required with a recursive definition to ensure that it actually 
defines something, and that what is defined makes sense. Recursion defines a 
concept in terms of itself, and we need to ensure that the definition is not circular 
(i.e. that it does not lead to a vicious circle). 

Recursion and induction are closely related and are often used together. 
Recursion is extremely useful in developing algorithms for solving complex 
problems, and induction is a useful technique in verifying the correctness of such 
algorithms. 


Example 4.1 Show that the sum of the first n natural numbers is given by the 
formula: 


n(n+ 1) 


7S 9h es seg 
. 2 


Proof Base Case 


We consider the case where n = | and clearly 1 = uit )) and so the base case P(1) 


is true. 
Inductive Step 


Suppose the result is true for some number k, then we have P(k) 


k(k +1) 


LP oth 


Then consider the sum of the first k + 1 natural numbers, and we use the 
inductive hypothesis to show that its sum is given by the formula: 


4.1 Introduction 81 


1424+3+-:-+k+(k+1) 
k(k+1 
- a det 1) (by inductive hypothesis) 
_R+k . (2k+2) 
= 7 
_ Wh +3k+2 
7 2, 
(+142) 


2, 
Thus, we have shown that if the formula is true for an arbitrary natural number &k, 
then it is true for its successor k + 1. That is, P(k) — P(k +1). We have shown that 


P(1) is true, and so it follows from mathematical induction that P(2), P(3), ... are 
true, and so P(n) is true, for all the natural numbers and the theorem is established. 


Note 4.1 There are opportunities to make errors in proofs with induction, and the 
most common mistakes are not to complete the base case or inductive step cor- 
rectly. These errors can lead to strange results and so care is required. It is important 
to be precise in the statements of the base case and inductive step. 


Example 4.2 (Binomial Theorem) Prove the binomial theorem using induction 
(permutations and combinations are discussed in Chap. 5). That is, 


(1+x)"=14 (T)x4 (S)tt et (Tee tae 


Proof Base Case 


We consider the case where n = 1 and clearly (1+x)'= (1+x) =1+4 ( )s 


and so the base case P(1) is true. 
Inductive Step 


Suppose the result is true for some number k, then we have P(k) 


(14+x)f=1+4+ ({)s+ (5 )?+ dha lcaht Gia geal al Ga 
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Then consider (1 + x)**! 


given by the formula: 


, and we use the inductive hypothesis to show that it is 


= k 
k+1 k+1 k+1\,, (k+1 

=1+ Xe + Meet xy a 
1 r k k+1 


(which follows from Exercise 7 below). 

Thus, we have shown that if the binomial theorem is true for an arbitrary natural 
number k, then it is true for its successor k + 1. That is, P(k) > P(k + 1). We have 
shown that P(1) is true, and so it follows from mathematical induction that P(n) is 
true, for all the natural numbers, and so the theorem is established. 

The standard formula of the binomial theorem (x+ y)" follows immediately 
from the formula for (1+x)", by noting that (x+y)"= {x(1+?/,)}"= 
x"(1+/,)". 


4.2 Strong Induction 


Strong induction is another form of mathematical induction, which is often 
employed when we cannot prove a result with (weak) mathematical induction. It is 
similar to weak induction in that there is a base step and an inductive step. The base 
step is identical to weak mathematical induction, and it involves showing that the 
statement is true for some natural number (usually the number 1). The inductive 
step is a little different, and it involves showing that if the statement is true for all 
the natural numbers less than or equal to an arbitrary number k, then the statement is 
true for its successor k+1. This is often written as (P(1) AP(2)A...A 
P(k)) > P(k+1). 
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From the base step and the inductive step, we infer that the statement is true for 
all the natural numbers (that are greater than or equal to the number specified in the 
base case). Formally, the proof technique used in mathematical induction is of the 
form’: 


(P(1) AVk|(P(1) A P(2) A... A P(k)) > P(k-+ 1)]) > Va P(n). 


Strong and weak mathematical induction are equivalent in that any proof done 
by weak mathematical induction may also be considered a proof using strong 
induction, and a proof conducted with strong induction may also be converted into 
a proof using weak induction. 

Weak mathematical induction is generally employed when it is reasonably clear 
how to prove P(k + 1) from P(k), with strong mathematical typically employed 
where it is not so obvious. The validity of both forms of mathematical induction 
follows from the well-ordering property of the Natural Numbers, which states that 
every non-empty set has a least element. 


Well-Ordering Principle 


Every non-empty set of natural numbers has a least element. The well-ordering 
principle is equivalent to the principle of mathematical induction. 


Example 4.3 Show that every natural number greater than one is divisible by a 
prime number. 


Proof Base Case 


We consider the case of n = 2 which is trivially true, since 2 is a prime number 
and is divisible by itself. 


Inductive Step (strong induction) 


Suppose that the result is true for every number less than or equal to k. Then we 
consider k + 1, and there are two cases to consider. If k + 1 is prime, then it is 
divisible by itself. Otherwise, it is composite and it may be factored as the product 
of two numbers each of which is less than or equal to k. Each of these numbers is 
divisible by a prime number by the strong inductive hypothesis, and so k + 1 is 
divisible by a prime number. 

Thus, we have shown that if all the natural numbers less than or equal to k are 
divisible by a prime number, then k + 1 is divisible by a prime number. We have 
shown that the base case P(2) is true, and so it follows from strong mathematical 
induction that every natural number greater than one is divisible by some prime 
number. 


*As before, this definition covers the base case of n = 1 and would need to be adjusted if the 
number specified in the base case is higher. 
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4.3 Recursion 


Some functions (or objects) used in mathematics (e.g. the Fibonacci sequence) are 
difficult to define explicitly, and are best defined by a recurrence relation: i.e. an 
equation that recursively defines a sequence of values, once one or more initial 
values are defined. Recursion may be employed to define functions, sequences and 
sets. 

There are two parts to a recursive definition, namely the base case and the 
recursive (inductive) step. The base case usually defines the value of the function at 
n=0 or n=1, whereas the recursive step specifies how the application of the 
function to a number may be obtained from its application to one or more smaller 
numbers. 

It is important that care is taken with the recursive definition, to ensure that that it 
is not circular, and does not lead to an infinite regress. The argument of the function 
on the right-hand side of the definition in the recursive step is usually smaller than 
the argument on the left-hand side to ensure termination (there are some unusual 
recursively defined functions such as the McCarthy 91 function where this is not the 
case). 

It is natural to ask when presented with a recursive definition whether it means 
anything at all, and in some cases the answer is negative. The fixed-point theory 
provides the mathematical foundations for recursion, and ensures that the 
functions/objects are well defined. 

Chapter 12 (Sect. 12.6) discusses various mathematical structures such as partial 
orders, complete partial orders and lattices, which may be employed to give a 
secure foundation for recursion. A precise mathematical meaning is given to 
recursively defined functions in terms of domains and fixed-point theory, and it is 
essential that the conditions in which recursion may be used safely be understood. 
The reader is referred to [1] for more detailed information. 

A recursive definition will include at least one non-recursive branch with every 
recursive branch occurring in a context that is different from the original, and brings 
it closer to the non-recursive case. Recursive definitions are a powerful and elegant 
way of giving the denotational semantics of language constructs. 

Next, we present examples of the recursive definition of the factorial function 
and Fibonacci numbers. 


Example 4.4 (Recursive Definition of Functions) The factorial function n! is very 
common in mathematics and its well-known definition is n! = 

n(n — 1)(n — 2)...3.2.1 and 0! = 1. The formal definition in terms of a base case 
and inductive step is given as follows: 


Base Step fac (0) = 1 
Recursive Step fac (n) =n * fac(n — 1) 


This recursive definition defines the procedure by which the factorial of a number is 
determined from the base case, or by the product of the number by the factorial of 
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its predecessor. The definition of the factorial function is built up in a sequence: fac 
(0), fac(1), fac(2), .... 

The Fibonacci sequence* is named after the Italian mathematician Fibonacci, who 
introduced it in the thirteenth century. It had been previously described in Indian 
mathematics, and the Fibonacci numbers are the numbers in the following integer 
sequence: 


1:1,9.3)5.8.19,01.54 


Each Fibonacci number (apart from the first two in the sequence) is obtained by 
adding the two previous Fibonacci numbers in the sequence together. Formally, the 
definition is given by 


Base Step F, =1,FR=1 
Recursive Step Fy, =Fn-1+F,-2 (Definition for when n > 2) 


Example 4.5 (Recursive Definition of Sets and Structures) Sets and sequences may 
also be defined recursively, and there are two parts to the recursive definition (as 
before). The base case specifies an initial collection of elements in the set, whereas 
the inductive step provides rules for adding new elements to the set based on those 
already there. Properties of recursively defined sets may often be proved by a 
technique called structural induction. 

Consider the subset S of the Natural Numbers defined by 


Base Step 5eS 
Recursive Step Forx € S thenx+5€S 


Then the elements in S are given by the set of all the multiples of 5, as clearly 
5 € S; therefore, by the recursive step 5+5 = 10 € 8;5+10= 15 € S; and so on. 
The recursive definition of the set of strings &* over an alphabet © is given by 


Base Step A € &* (Lis the empty string) 
Recursive Step Foro € X*andv € Xthen ov € X* 


Clearly, the empty string is created from the base step. The recursive step states that 
a new string is obtained by adding a letter from the alphabet to the end of an 
existing string in &*. Each application of the inductive step produces a new string 
that contains one additional character. For example, if X = {0, 1} then the strings in 
&* are the set of bit strings A, 0, 1, 00, 01, 10, 11, 000, 001, 010, etc. 

We can define an operation to determine the length of a string (len: &* — N) 
recursively. 


3We are taking the Fibonacci sequence as starting at 1, whereas others take it as starting at 0. 
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Base Step len (A) =0 
Recursive Step len(av) =len(o) +1 (whereo € X* andv € x) 


A binary tree’ is a well-known data structure in computer science, and it consists of 
a root node together with a left and right binary tree. A binary tree is defined as a 
finite set of nodes (starting with the root node), where each node consists of a data 
value and a link to a left subtree and a right subtree. Recursion is often used to 
define the structure of a binary tree. 

Base Step A single node is a binary tree (root) 

Recursive Step 


(i) Suppose X and Y are binary trees and x is a node then XxY is a 
binary tree, where X is the left subtree, Y is the right subtree and 
x is the new root node. 

(ii) Suppose X is a binary tree and x is a node then xX and Xx are 
binary trees, which consist of the root node x and a single child 
left or right subtree. 


That is, a binary tree has a root node and it may have no subtrees; it may consist of a 
root node with a left subtree only; a root node with a right subtree only; or a root 
node with both a left and a right subtree. 


4.4 Structural Induction 


Structural induction is a mathematical technique that is used to prove properties 
about recursively defined sets and structures. It may be used to show that all the 
members of a recursively defined set have a certain property, and there are two parts 
to the proof (as before), namely the base case and the recursive (inductive) step. 

The first part of the proof is to show that the property holds for all the elements 
specified in the base case of the recursive definition. The second part of the proof 
involves showing that if the property is true for all the elements used to construct 
the new elements in the recursive definition then the property holds for the new 
elements. From the base case and the recursive step, we deduce that the property 
holds for all the elements of the set (structure). 


Example 4.6 (Structural Induction) We gave a recursive definition of the subset 
S of the natural numbers that consists of all the multiples of 5. We did not prove that 
all the elements of the set S are divisible by 5, and we use structural induction to 
prove this. 


“We will give an alternate definition of a tree in terms of a connected acyclic graph in Chap. 9 on 
graph theory. 
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Base Step 5 € S (and clearly the base case is divisible by 5). 

Inductive Step Suppose q € S then q = 5 k for some k. From the inductive 
hypothesis, g+ 5 € Sandg+5 =5k+5 = 5(k+1) andsog + Sis 
divisible by 5. 
Therefore, all the elements of S are divisible by 5. 


4.5 Review Questions 


Show that 9"+ 7 is always divisible by 8. 

Show that the sum of 17+ 27+ --- +n? =n(n+1)(2n+1)/6 

Explain the difference between strong and weak induction. 

What is structural induction? 

Explain how recursion is used in mathematics. 

Investigate the recursive definition of the McCarthy 91 function, and explain 
how it differs from usual recursive definitions. 


q, Show that (7) + ( y J-(P). 
r r—1 r 


8. Determine the standard formula for the binomial theorem (x+y)" from the 
formula for (1 +x)”. 


De ee 


4.6 Summary 


Mathematical induction is an important proof technique that is used to establish the 
truth of a statement for all the natural numbers. There are two parts to a proof by 
induction, and these are the base case and the inductive step. The base case involves 
showing that the statement is true for some natural number (usually for the number 
n = 1). The inductive step involves showing that if the statement is true for some 
natural number n = k, then the statement is true for its successor n = k+ 1. 

From the base step and the inductive step, we infer that the statement is true for 
all the natural numbers (that are greater than or equal to the number specified in the 
base case). Mathematical induction may be used to prove a wide variety of theo- 
rems, such as theorems about summation formulae, inequalities, set theory, and the 
correctness of algorithms and computer programs. 

Strong induction is often employed when we cannot prove a result with (weak) 
mathematical induction. It also has a base case and an inductive step, where the 
inductive step is a little different, and it involves showing that if the statement is true 
for all the natural numbers less than or equal to an arbitrary number k, then the 
statement is true for its successor k + 1. 
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Recursion may be employed to define functions, sequences and sets in mathe- 
matics, and there are two parts to a recursive definition, namely the base case and 
the recursive step. The base case usually defines the value of the function at n = 0 
orn = 1, whereas the recursive step specifies how the application of the function to 
a number may be obtained from its application to one or more smaller numbers. It is 
important that care is taken with the recursive definition, to ensure that that it is not 
circular, and does not lead to an infinite regress. 

Structural induction is a mathematical technique that is used to prove properties 
about recursively defined sets and structures. It may be used to show that all the 
members of a recursively defined set have a certain property, and there are two parts 
to the proof, namely the base case and the recursive (inductive) step. 


Reference 
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Sequences, Series, and Permutations 5 
and Combinations 


5.1. Introduction 


The goal of this chapter is to provide an introduction to sequences and series, 
including arithmetic and geometric sequences, and arithmetic and geometric series. 
We derive formulae for the sum of an arithmetic series and geometric series, and we 
discuss the convergence of a geometric series when |r| <1, and the limit of its sum 
as n gets larger and larger. 

We discuss the calculation of simple and compound interest, and the concept of the 
time value of money, and its application to determine the present value of a payment to 
be made in the future. We then discuss annuities, which are a series of payments made 
at regular intervals over a period of time, and we determine the present value of an 
annuity. Basic financial mathematics is discussed in more detail in Chap. 25. 

We consider the counting principle, where one operation has m possible out- 
comes and a second operation has n possible outcomes. We determine that the total 
number of outcomes after performing the first operation followed by the second 
operation to be m x n. A permutation is an arrangement of a given number of 
objects, by taking some or all of them at a time. The order of the arrangement is 
important, as the arrangement ‘abc’ is different from ‘cba’. A combination is a 
selection of a number of objects in any order, where the order of the selection is 
unimportant. That is, the selection ‘abc’ is the same as the selection ‘cba’. 


5.2 Sequences and Series 


A sequence a), a2, .... dy, ... is any succession of terms (usually numbers), and we 
discussed the Fibonacci sequence earlier in Chap. 4. Each term in the Fibonacci 
sequence (apart from the first two terms) is obtained from the sum of the previous 
two terms in the sequence: 


11,9, 5, 5,813, 21,243 
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A sequence may be finite (with a fixed number of terms) or infinite. The 
Fibonacci sequence is infinite whereas the sequence 2, 4, 6, 8, 10 is finite. We 
distinguish between convergent and divergent sequences, where a convergent 
sequence approaches a certain value as n gets larger and larger (technically, we say 
that ,... lima, exists (i.e. the limit of a, exists)). Otherwise, the sequence is said to 
be divergent. 

Often, there is a mathematical expression for the nth term in a sequence (e.g. for the 
sequence of even integers 2, 4, 6, 8, ... the general expression for a, is given by 
a, = 2n). Clearly, the sequence of the even integers is divergent, as it does not approach 
a particular value, as n gets larger and larger. Consider the following sequence: 


1,-1,1, -1,1 


a at a ,al 
Then this sequence is divergent since it does not approach a certain value, as 
n gets larger and larger, since it continues to alternate between 1 and —1. The 


formula for the nth term in the sequence may be given by. 
e 1 er 1 


The sequence 1, %, '/3, 4, ... '/, ... is convergent and it converges to 0. The nth 
term in the sequence is given by '/,, and as n gets larger and larger, it gets closer 
and closer to 0. 

A series is the sum of the terms in a sequence, and the sum of the first m terms of 
the sequence dj, a3, .... d, ... is given by aj +a2+ --+ +a,, which is denoted by 


n 
>a 


k=1 


A series is convergent if its sum approaches a certain value S as n gets larger and 
larger, and this is written formally as 


n 
lim ) a, =S. 
n—00 

k=1 


Otherwise, the series is said to be divergent. 


5.3 Arithmetic and Geometric Sequences 


Consider the sequence 1, 4, 7, 10, ... where each term is obtained from the previous 
term by adding the constant value 3. This is an example of an arithmetic sequence, 
and there is a difference of 3 between any term and the previous one. The general 
form of a term in this sequence is a, = 3n — 2. 
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The general form of an arithmetic sequence is given by 
a,a+d,a+2d,a+3d,...a+(n-—1)d,... 


The value a is the initial term in the sequence, and the value d is the constant 
difference between a term and its successor. For the sequence, 1, 4, 7, .., we have 
a= 1 and d = 3, and the sequence is not convergent. In fact, all the arithmetic 


sequences (apart from the constant sequence a, a, ..., a which converges to a) are 
divergent. 
Consider, the sequence 1, 3, 9, 27, 81, ..., where each term is achieved from the 


previous term by multiplying by the constant value 3. This is an example of a 
geometric sequence, and the general form of a geometric sequence is given by 
a,ar,ar’,ar’, ..., ar"! 

The first term in the geometric sequence is a and r is the common ratio. Each 
term is obtained from the previous one by multiplying by the common ratio r. For 
the sequence 1, 3, 9, 27 the value of a is | and r is 3. 

A geometric sequence is convergent if |r| <1, and for this case it converges to 0. 
It is also convergent if r = 1, as for this case it is simply the constant sequence a, a, 


a, ...., which converges to a. For the case where |r| > 1 the sequence is divergent, 
and the sequence alternates between a and —a for r = —1. 


5.4 Arithmetic and Geometric Series 


An arithmetic series is the sum of the terms in an arithmetic sequence, and a 
geometric sequence is the sum of the terms in a geometric sequence. It is possible to 
derive a simple formula for the sum of the first n terms in arithmetic and geometric 
series. 


Arithmetic Series 


We write the series in two ways: first, the normal left to right addition, and then the 
reverse, and then we add both series together: 


Sn =a+(a+d)+(a+2d) + (a+3d) +--+ +(a+(n— 1)d) 
Sn=a+(n—1)d+a+(n—2)d+---+ +(a+d)+a 
2Sn = [2a+(n — 1)d]+ [2a+ (n— 1)d]+---+[2a+(n—1)d] (ntimes) 


2Sn =n x [2a+(n— 1)d]. 


92 5 Sequences, Series, and Permutations and Combinations 


Therefore, we conclude that 


Sn = 3 a+ (n — 1)d}. 


Example (Arithmetic Series) 
Find the sum of the first m terms in the following arithmetic series 1, 3, 5, 7, 9... 
Solution 


Clearly, a = 1 and d = 2. Therefore, applying the formula, we get 


2 
n 2n 
S, =~[2.1+ (n— 1)2] =—=n’. 
2 2 
Geometric Series 
For a geometric series we have 
S, = a+ar+ar +ar 4 tar"! 
=>7rS,= ar+ar +ar 4 tar’! +ar 


Therefore, we conclude that (where r 4 1) 


ce) Se) 


r—1l l-r 


The case when r = 1| corresponds to the arithmetic series a + a+ +--+ a, and 
the sum of this series is simply na. The geometric series converges when |r|< 1 as 
r” + 0 asn—«, and so 


Sn asn x. 


l-r 


Example (Geometric Series) 


Find the sum of the first n terms in the following geometric series 1, ya Pe Me ia 
What is the sum of the series? 


Solution 


Clearly, a = 1 and r = '/>. Therefore, applying the formula, we get 
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ee oe ee 
=a = fa 


The sum of the series is the limit of the sum of the first n terms as n approaches 
infinity. This is given by 


lim S, = lim 2(1 — 14") = 2. 


nox nave 


5.5 Simple and Compound Interest 


Savers receive interest on placing deposits at the bank for a period of time, whereas 
lenders pay interest on their loans to the bank. We distinguish between simple and 
compound interest, where simple interest is always calculated on the original 
principal, whereas for compound interest, the interest is added to the principal sum, 
so that interest is also earned on the added interest for the next compounding period. 

For example, if Euro 1000 is placed on deposit at a bank with an interest rate of 
10% per annum for 2 years, it would earn a total of Euro 200 in simple interest. The 
interest amount is calculated by 


1000 «10 «2 


100 = Euro 200. 


The general formula for calculating simple interest on principal P, at a rate of 
interest J, and for time T (in years:), is 
_PxIxT 
~ 100 


A 


The calculation of compound interest is more complicated as may be seen from 
the following example. 


Example (Compound Interest) 


Calculate the interest earned and what the new principal will be on Euro 1000, 
which is placed on deposit at a bank, with an interest rate of 10% per annum 
(compound) for 3 years. 


Solution 


At the end of year 1, Euro 100 of interest is earned, and this is capitalized making 
the new principal at the start of year 2 Euro 1100. At the end of year 2, Euro 110 is 
earned in interest, making the new principal at the start of year 3 Euro 1210. 
Finally, at the end of year 3, a further Euro 121 is earned in interest, and so the new 
principal is Euro 1331 and the total interest earned for the 3 years is the sum of the 
interest earned for each year (i.e. Euro 331). This may be seen from Table 5.1. 
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Table 5.1 Calculation of 


. Year Principal Interest earned 
compound interest 1000 100 
2 1100 110 
3 1210 121 


The new principal each year is given by the geometric sequence with a = 1000 
and r = 10/100 = 0.1. 


1000, 1000(1.1), 1000(1.1)”, 1000(1.1)°, ... 


In general, if a principal amount P is invested for T years at a rate R of interest 
(r is expressed as a proportion, i.e. r = =‘ 100), then it will amount to 


A=P(1+nr)’. 
For our example above, A = 1000, T = 3 and r = 0.1 Therefore, 


A = 1000 (1.1)? 
= 1331(as before). 


There are variants of the compound interest formula to cover situations where 
there are m-compounding periods per year, and so the reader may consult the 
available texts. We discuss basic financial mathematics in more detail in Chap. 25. 


5.6 Time Value of Money and Annuities 


The time value of money discusses the concept that the earlier that cash is received 
the greater value it has to the recipient. Similarly, the later that a cash payment is 
made, the lower its value to the recipient, and the lower its cost to the payer. 

This is clear if we consider the example of a person who receives $1000 now and 
a person who receives $1000 5 years from now. The person who receives $1000 
now is able to invest it and to receive annual interest on the principal, whereas the 
other person who receives $1000 in 5 years earns no interest during the period. 
Further, the inflation during the period means that the purchasing power of $1000 is 
less in 5 years time is less than it is today. 

We presented the general formula for what the future value of a principal P invested 
for n years at a compound rate r of interest is A = P(1 +7)". We can determine the 
present value of an amount A received in n years time at a discount rate r by 


A 


Page 
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An annuity is a series of equal cash payments made at regular intervals over a 
period of time, and so there is a need to calculate the present value of the series of 
payments made over the period. The actual method of calculation is clear from 
Table 5.2. 


Example (Annuities) 


Calculate the present value of a series of payments of $1000 (made at the end of 
each year) with the payments made for 5 years at a discount rate of 10%. 


Solution 


The regular payment A is 1000, the rate r is 0.1 and n = 5. The present value of the 
payment received at the end of year | is 1000/1.1 = 909.91; at the end of year 2 it is 
1000/(1.1)” = 826.45; and so on. The total present value of the payments over the 5 
years is given by the sum of the individual present values and is $3791 (Table 5.2). 

We may easily derive a formula for the present value of a series of payments 
A over a period of n years at a discount rate of r as follows: Clearly, the present 
value is given by 


A A A 


(En ee Ce 


This is a geometric series where the constant ratio is es and the present value of 


the annuity is given by its sum: 


pv =“11 : 
=7ll-qgye 


For the example above, we apply the formula and get 


0.1 (1.1)? 
= 10000 (0.3791) 
= $3791. 


Financial mathematics is discussed in more detail in Chap. 25. 


Table 5.2 Calculation of 


: Year Amount Present value (r = 0.1) 
present value of annuity 
1 1000 909.91 
a 1000 826.44 
3 1000 751.31 
4 1000 683.01 
5 1000 620.92 
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5.7. Permutations and Combinations 


A permutation is an arrangement of a given number of objects, by taking some or 
all of them at a time. A combination is a selection of a number of objects where the 
order of the selection is unimportant. Permutations and combinations are defined in 
terms of the factorial function, which was defined in Chap. 4. 


Principles of Counting 


(a) Suppose one operation has m possible outcomes and a second operation has 
n possible outcomes, then the total number of possible outcomes when per- 
forming the first operation followed by the second operation ism x n (Product 
Rule). 

(b) Suppose one operation has m possible outcomes and a second operation has 
n possible outcomes, then the total number of possible outcomes of the first 
operation or the second operation is given by m + n (Sum Rule). 


Example (Counting Principle (a)) 


Suppose a dice is thrown and a coin is then tossed. How many different outcomes 
are there and what are they? 


Solution 


There are six possible outcomes from a throw of the dice, 1, 2, 3, 4, 5 or 6, and there 
are two possible outcomes from the toss of a coin, H or T. Therefore, the total 
number of outcomes is determined from the product rule as 6 x 2 = 12. The 
outcomes are given by 


(1,H), (2,H), (3,H), (4,H), (5,H), (6,H), (1,T), (2,T), (3,T), (4,T), (5,T), (6,T). 


Example (Counting Principle (5)) 


Suppose a dice is thrown and if the number is even a coin is tossed and if it is odd 
then there is a second throw of the dice. How many different outcomes are there? 


Solution 


There are two experiments involved with the first experiment involving an even 
number and a toss of a coin. There are 3 possible outcomes that result in an even 
number and 2 outcomes from the toss of a coin. Therefore, there are 3 x 2 = 6 
outcomes from the first experiment. 

The second experiment involves an odd number from the throw of a dice and the 
further throw of the dice. There are 3 possible outcomes that result in an odd 
number and 6 outcomes from the throw of a dice. Therefore, there are 3 x 6 = 18 
outcomes from the second experiment. 
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Finally, there are 6 outcomes from the first experiment and 18 outcomes from the 
second experiment, and so from the sum rule there are a total of 6 + 18 = 24 
outcomes. 


Pigeonhole Principle 


The pigeonhole principle states that if n items are placed into m containers (with 
n >m), then at least one container must contain more than one item (Fig. 5.1). 


Examples (Pigeonhole Principle) 


(a) Suppose there is a group of 367 people, then there must be at least two people 
with the same birthday. 


This is clear as there are 365 days in a year (with 366 days in a leap year), and 
so as there are at most 366 possible birthdays in a year. The group size is 367 
people, and so there must be at least two people with the same birthday. 

(b) Suppose that a class of 102 students is assessed in an examination (the outcome 

from the exam is a mark between 0 and 100). Then, there are at least two 
students who receive the same mark. 
This is clear as there are 101 possible outcomes from the test (as the mark that a 
student may achieve is between is between 0 and 100), and as there are 102 
students in the class and 101 possible outcomes from the test, then there must 
be at least two students who receive the same mark. 


Fig. 5.1 Pigeonhole principle 
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Permutations 


A permutation is an arrangement of a number of objects in a definite order. 
Consider the three letters A, B and C. If these letters are written in a row, then 
there are six possible arrangements: 


ABC, ACB, BAC, BCA, CAB or CBA. 


There is a choice of 3 letters for the first place, then there is a choice of 2 letters 
for the second place and there is only 1 choice for the third place. Therefore, there 
are 3 x 2 x 1 = 6 arrangements. 

If there are n different objects to arrange, then the total number of arrangements 
(permutations) of n objects is given by n! = n(n — 1)(m — 2)... 3.2.1. 

Consider the four letters A, B, C and D. How many arrangements (taking 2 
letters at a time with no repetition) of these letters can be made? 

There are 4 choices for the first letter and 3 choices for the second letter, and so 
there are 12 possible arrangements. These are given by 


AB, AC, AD, BA,BC, BD,CA,CB,CD,DA, DB or DC. 


The total number of arrangements of n different objects taking r at a time 
(r < n)is given by "P, = n(n — 1)(n — 2)...(n — r+ 1). It may also be written as 


Example (Permutations) 


Suppose A, B, C, D, E and F are six students. How many ways can they be seated 
in a row if 


(i) There is no restriction on the seating. 
Gi) A and B must sit next to one another. 
(ii) A and B must not sit next to one another. 


Solution 


For unrestricted seating, the number of arrangements is given by 
6.5.4.3.2.1 = 6! = 720. 

For the case where A and B must be seated next to one another, then consider A 
and B as one person, and then the five people may be arranged in 5! = 120 ways. 
There are 2! =2 ways in which AB may be arranged, and so there are 
2! x 5! = 240 arrangements: 


AB C D E F 
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For the case where A and B must not be seated next to one another, then this is 
given by the difference between the total number of arrangements and the number 
of arrangements with A and B together: i.e. 720 —240 = 480. 


Combinatations 


A combination is a selection of a number of objects in any order, and the order of 
the selection is unimportant, in that both AB and BA represent the same selection. 
The total number of arrangements of n different objects taking r at a time is given 
by "P,, and we can determine that the number of ways that r objects can be selected 
from n different objects from this, as each selection may be permuted r! times, and 
so the total number of selections is r! x total number of combinations. That is, 
"P.=r! x" C, and we may also write this as 


(") nl n(n —1)...(n—r+1) 


r 


ri(n—r)! r! 


It is clear from the definition that 


Example 1 (Combinations) 


How many ways are there to choose a team of 11 players from a panel of 15 
players? 


Solution 


Clearly, the number of ways is given by ( i = ( a) 
That is, 15.14.13.12/4.3.2.1 = 1365. 
Example 2 (Combinations) 


How many ways can a committee of 4 people be chosen from a panel of 10 people 
where 


(i) There is no restriction on membership of the panel. 
(ii) A certain person must be a member. 
(iii) A certain person must not be a member. 


Solution 
For (i) with no restrictions on membership, the number of selections of a committee 


of 4 people from a panel of 10 people is given by ( ») = 210. 


100 
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For (ii) where one person must be a member of the committee, then this involves 


choosing 3 people from a panel of 9 people and is given by (5) = 84. 


For (iii) where one person must not be a member of the committee, then this 


involves choosing 4 people from a panel of 9 people, and is given by (4) = 126. 


5.8 


Review Questions 


. Determine the formula for the general term and the sum of the following 


arithmetic sequence 


4 oe 


. Write down the formula for the nth term in the following sequence: 


1y,! on oe / 08> snes 


. Find the sum of the following geometric sequence: 


1 1 
[3 Vien ia 


. How many years will it take a principal of $5000 to exceed $10,000 at a 


constant annual growth rate of 6% compound interest? 


. What is the present value of $5000 to be received in 5 years’ time at a 


discount rate of 7%? 


. Determine the present value of a 20-year annuity of an annual payment of 


$5000 per year at a discount rate of 5%. 


. How many different five-digit numbers can be formed from the digits 1, 2, 


3, 4, 5 where 


(i) No restrictions on digits and repetitions are allowed. 
Gi) The number is odd and no repetitions are allowed. 
(iii) The number is even and repetitions are allowed. 
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8. (i) How many ways can a group of five people be selected from nine 
people? 
(ii) How many ways can a group be selected if two particular people 
are always included? 
(iii) How many ways can a group be selected if two particular people 
are always excluded? 


5.9 Summary 


This chapter provided a brief introduction to sequences and series, including 
arithmetic and geometric sequences, and arithmetic series and geometric series. We 
derived formulae for the sum of an arithmetic series and geometric series, and we 
discussed the convergence of a geometric series when |r| <1. 

We discussed the calculation of simple and compound interest, and the concept 
of the time value of money, and its application to determine the present value of a 
payment to be made in the future. We discussed annuities, which are a series of 
payments made at regular intervals over a period of time, and we calculated the 
present value of an annuity. 

We considered counting principles including the product and sum rules. The 
product rule is concerned with where one operation has m possible outcomes and a 
second operation has n possible outcomes, then the total number of possible out- 
comes when performing the first operation followed by the second operation is 
mxXn. 

We discussed the pigeonhole principle, which states that if n items are placed 
into m containers (with n > m), then at least one container must contain more than 
one item. We discussed permutations and combinations where permutations are an 
arrangement of a given number of objects, by taking some or all of them at a time. 
A combination is a selection of a number of objects in any order, and the order of 
the selection is unimportant. 
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Algebra 


6.1. Introduction 


Algebra is the branch of mathematics that uses letters in the place of numbers, 
where the letters stand for variables or constants that are used in mathematical 
expressions. Algebra is the study of such mathematical symbols and the rules for 
manipulating them, and it is a powerful tool for problem-solving in science and 
engineering. 

The origins of algebra are in the work done by Islamic mathematicians during 
the Golden age in Islamic civilization, and the word ‘algebra’ comes from the 
Arabic term ‘al-jabr’, which appears as part of the title of a book by the Islamic 
mathematician, Al-Khwarizmi, in the ninth century A.D. The third century A.D. 
Hellenistic mathematician, Diophantus, also did early work on algebra, and we 
mentioned in Chap. | that the Babylonians employed an early form of algebra. 

Algebra covers many areas such as elementary algebra, linear algebra and 
abstract algebra. Elementary algebra includes the study of symbols and rules for 
manipulating them to form valid mathematical expressions, simultaneous equations, 
quadratic equations, polynomials, indices and logarithms. Linear algebra is con- 
cerned with the solution of a set of linear equations, and includes the study of 
matrices (see Chap. 8) and vectors. Abstract algebra is concerned with the study of 
abstract algebraic structures such as monoids, groups, rings, integral domains, fields 
and vector spaces. 


6.2 Simple and Simultaneous Equations 


A simple equation is an equation with one unknown, and the unknown may be on 
both the left-hand and right-hand sides of the equation. The method of solving such 
equations is to bring the unknowns to one side of the equation, and the values to the 
other side. 
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Simultaneous equations are equations with two (or more) unknowns. There are a 
number of methods to finding a solution to two simultaneous equations such as 
elimination, substitution and graphical techniques. The solution to a set of n linear 
equations with n unknowns may be done using Gaussian elimination and matrix 
theory (see Chap. 8). 


Example (Simple Equation) 
Solve the simple equation 4 — 3x = 2x — 11. 


Solution (Simple Equation) 


4-—3x= 2x-11 
4—(—11) = 2x—(3x) 
4+11=2x+3x 
15 = 5x 

3x. 


Example (Simultaneous Equation—Substitution Method) 
Solve the following simultaneous equations by the method of substitution: 


x+2y=-1 
4x — 3y = 18. 


Solution (Simultaneous Equation—Substitution Method) 


The method of substitution involves expressing x in terms of y and substituting it in 
the other equation (or vice versa expressing y in terms of x and substituting it in the 
other equation). For this example, we use the first equation to express x in terms of 
y: 


x+2y=-1 
x=-1-2y. 


We then substitute for x (—1 — 2y) in the second equation, and we get a simple 
equation involving just the unknown y: 


4(—1 — 2y) — 3y = 18 
=> —4-8y—3y = 18 
=> -—-lly=18+4 
=> -lly = 22 
>y=-2. 
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We then obtain the value of x from the substitution: 


x=-l1-2y 

=> x = —1—2(-2) 
>x=-14+4 
=>x=3. 


We can then verify that our solution is correct by checking our answer for both 
equations: 


Example (Simultaneous Equation—Method of Elimination) 
Solve the following simultaneous equations by the method of elimination: 


3x+4y = 5 
2x—5y = —12. 


Solution (Simultaneous Equation—Method of Elimination) 

The approach is to manipulate both equations so that we may eliminate either 
x or y, and so reduce to a simple equation with just x or y. For this example, we are 
going to eliminate x, and so we multiply Eq. (1) by 2 and Eq. (2) by —3 and this 
yields two equations with the opposite coefficient of x: 


6x + 8y = 10 

— 6x+ 15y = 36 

Ox + 23y = 46 
y= 2. 


We then add both equations together and conclude that y = 2. We then deter- 
mine the value of x by replacing y with 2 in the first equation: 


3x+4(2) =5 
3x+8=5 
3x = 5-8 
3x = -3 


x=-l. 
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We can then verify that our solution is correct as before by checking our answer 
for both equations. 


Example (Simultaneous Equation—Graphical Techniques) 


Find the solution to the following simultaneous equations using graphical 
techniques: 


x+2y=-1 
4x — 3y = 18. 


Solution (Simultaneous Equation—Graphical Techniques) 

Each simultaneous equation represents a straight line, and so the solution to the 
two simultaneous equations is the point of intersection of both lines (if there is such 
a point). Therefore, the solution involves drawing each line and finding the point of 
intersection of both lines (Fig. 6.1). 

First, we find two points on line 1: e.g. (0, —0.5) and (—1, 0) are on line 1, since 
when x = 0 we have 2y = —1 and so y = —0.5. Similarly, when y = 0 we have 
x = —1. Next, we find two points on line 2 in a similar way: e.g. when x is 0 y is -6, 
and when y is 0 we have x = 4.5 and so the points (0, —6) and (4.5, 0) are on line 2. 

We then draw the X-axis and the Y-axis, draw the scales on the axes, label the 
axes, plot the points and draw both lines. Finally, we find the point of intersection 
of both lines (if there is such a point), and this is our solution to the simultaneous 
equations. 


solution 


Fig. 6.1 Graphical solution to simultaneous equations 
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For this example, there is a point of intersection for the lines, and so we 
determine the x and y coordinates, and the solution is then given by x = 3 and 
y = —2. The solution using graphical techniques requires care (as inaccuracies may 
be introduced from poor drawing) and graph paper is required for accuracy. 


6.3 Quadratic Equations 


A quadratic equation is an equation of the form ax? + bx +c = 0, and solving the 
quadratic equation is concerned with finding the unknown value x (roots of the 
quadratic equation). There are several techniques to solve quadratic equations such 
as factorization; completing the square, the quadratic formula and graphical 
techniques. 


Example (Quadratic Equations—Factorization) 
Solve the quadratic equation 3x*—11x—4 = 0 by factorization. 
Solution (Quadratic Equations—Factorization) 


The approach taken is to find the factors of the quadratic equation. Sometimes this 
is easy, but often other techniques will need to be employed. For the above 
quadratic equation, we note immediately that its factors are (3x + 1)(x — 4) since 


(3x+ 1)(x — 4) 
= 3x7—12x+x-4 
= 3x7—11x—4. 
Next, we note the property that if the product of two numbers A and B is 0 then 


either A is O or B is 0. In other words, AB =0 = > A =OorB =O. We conclude 
from this property that as 


3x7-11x-4 = 0, 


(3x+ 1)(x—4) =0 

(3x+ 1) = 0 or (x—4) = 0 
3x =—-lorx=4 

x = —0.33 orx = 4. 


Therefore, the solution (or roots) of the quadratic equation 3x7—11x—4 = 0 is 
x = —0.33 or x = 4. 


Example (Quadratic Equations—Completing the Square) 


Solve the quadratic equation 2x* + 5x—3 = 0 by completing the square. 
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Solution (Quadratic Equations—Completing the Square) 


First, we convert the quadratic equation to an equivalent quadratic with a unary 
coefficient of x2. This involves division by 2. Next, we examine the coefficient of x 
(in this case 5/2), and we add the square of half the coefficient of x to both sides. 
This allows us to complete the square, and we then take the square root of both 
sides. Finally, we solve for x. 


2x* + 5x3 = 0, 


ex fea =0 
© P45 /ox=? /y 

4 5/oxt C/4) = lat Cla) 
x+5/,)'=3 /y+ (°/16) 
#49), ligt (°/16) 
+54) = ho 


Example 1 (Quadratic Equations—Quadratic Formula) 
Establish the quadratic formula for solving quadratic equations. 
Solution (Quadratic Equations—Quadratic Formula) 


We complete the square and the result will follow. 


ax’ +bx+c =0, 
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Example 2 (Quadratic Equations—Quadratic Formula) 
Solve the quadratic equation 2x” + 5x—3 = 0 using the quadratic formula. 
Solution (Quadratic Equations—Quadratic Formula) 


For this example, a = 2, b=5 and c = —3, and we put these values into the 
quadratic formula: 


pe DE VY = 42(-3) _ -5 + V25 +24 
2.2 4 

_—-54V49  -547 

a er 


Example (Quadratic Equations—Graphical Techniques) 


Solve the quadratic equation 2x — x—6 = 0 using graphical techniques given that 
the roots of the quadratic equation lie between x = —3 and x = 3. 


Solution (Quadratic Equations—Graphical Techniques) 


The approach is first to create a table of values (Table 6.1) for the curve 
y= 2x? — x—6, and to draw the X- and Y-axis and scales, and then to plot the 
points from the table of values, and to join the points together to form the curve 
(Fig. 6.2). 

The graphical solution is to the quadratic equation is then given by the points 
where the curve intersects the X-axis (i.e. y = 0 on the X-axis). There may be no 
solution (i.e. the curve does not intersect the X-axis), one solution (a double root) or 
two solutions. 

The graph for the curve y = 2x* — x—6 is given below, and so the points where 
the curve intersects the X-axis are determined. We note from the graph that the 
curve intersects the X-axis at two distinct points, and we see from the graph that the 
roots of the quadratic equation are given by x = —1.5 and x = 2. 

The solution to quadratic equations using graphical techniques requires care (as 
with the solution to simultaneous equations using graphical techniques), and graph 
paper is required for accuracy. 


6.4 Indices and Logarithms 


The product a+ a*a+a... a(n times) is denoted by a”, and the number 7 is the 
index of a. The following are properties of indices: 
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Fig. 6.2. Graphical solution to quadratic equation 


apes oh Table of values X = ~2 =i 0 1 2 
’ ti t 
or Quadratic equation pees 15 4 3 ae 5 0 
a=i1 
m+n = q" a’ 
qu aq” n 
1 
WO eee 
a a 
1 n 
—=> da. 
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Logarithms are closely related to indices, and if the number b can be written in 
the form b=a", then we say that log to the base a of b is x: ie. 
log,b =x = a‘ =b. Clearly, log;g 100 = 2 since 10° = 100. The following are 
properties of logarithms: 


log, AB = log, A+ log, B 
log, A” =n log, A 


A 
logs = logA — log B. 
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We will prove the first property of logarithms. Suppose log, A = x and log, 
Bz=y. Then A=a“ and B=a and so AB=a'‘a’=a‘*? and so 
log, AB = x+y = log, A+ log, B. 

The law of logarithms may be used to solve certain equations involving powers 
(called indicial equations). We illustrate this by an example. 


Example (Indicial Equations) 
Solve the equation 2“ = 3 correct to 4 significant places. 
Solution (Indicial Equations) 
2* =3 
= log) 2* = log, 3 
=x logi9 2 = logy, 3 


6.5 Horner’s Method for Polynomials 


Horner’s method is a computationally efficient way to evaluate a polynomial 
function. It is named after William Horner who was a nineteenth-century British 
mathematician and schoolmaster. Chinese mathematicians were familiar with the 
method in the third century A.D. 

The normal method for the evaluation of a polynomial involves computing 
exponentials, and this is computationally expensive. Horner’s method has the 
advantage that fewer calculations are required, and it eliminates all exponentials by 
using nested multiplication and addition. It also provides a computationally efficient 
way to determine the derivative of the polynomial. 


Horner’s Method and Algorithm 
Consider a polynomial P(x) of degree n defined by 


P(x) = apx" + yx) 4 dy_ox 7 + Fayx+ao. 
The Horner method to evaluate P(xo) essentially involves writing P(x) as 


P(x) = (((anX + An—1)X + An—2)x + oars +a)x+ao. 
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The computation of P(xo) involves defining a set of coefficients b; such that 


Dn = 4n 


Dn-1 = Gn-1 bnX0 


by = ae + DK 1X0 


bj =a, +b2x9 


bop = ao + bX. 
Then the computation of P(xo) is given by 
P(xo) = bo. 
Q(x) = Dyx"! + By x"? + Dy_ox"-3 +..... +1, then it is easy to verify that 
P(x) = (x—x0)O(x) + do. 
This also allows the derivative of P(x) to be easily computed for xp since 
PI(x) = Q(x) + (x—x0) Q"(x) 
PI(xo) = Q(x0). 
Algorithm (To evaluate polynomial and its derivative) 


(i) Initialize y to a, and z to a, (compute b, for P and b,_, for Q); 
(ii) For each j from n — 1, n — 2 to 1, compute b; for P and b,-; for Q by 
setting y to xoy + a; Ge. b; for P) and z to xoz+ y (ie. b;-; for Q). 
(iii) Compute bo by setting y to xyy + ao. 


Then P(x9) = y and Pi(xo) = z. 


6.6 Abstract Algebra 


One of the important features of modern mathematics is the power of the abstract 
approach. This has opened up whole new areas of mathematics, and it has led to a 
large body of new results and problems. The term ‘abstract’ is subjective, as what is 
abstract to one person may be quite concrete to another. We shall introduce some 
important algebraic structures in this section including monoids, groups, rings, 
fields and vector spaces. 
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6.6.1 Monoids and Groups 


A non-empty set M together with a binary operation ‘*’ is called a monoid if for all 
elements a, b, c € M the following properties hold: 


()axbeEM (Closure property) 
(2) ax (bx c) = (axb)*c (Associative property) 
(3) Su € M such thata*u=uxa=a(Va eM). (Identity Element) 


A monoid is commutative if a * b = b* a for alla, b € M. A semi-group (M, *) 
is a set with a binary operation ‘*’ such that the closure and associativity properties 
hold (but it may not have an identity element). 


Example 6.1 (Monoids) 


(i) The set of sequences © under concatenation with the empty sequence A is the 
identity element. 

(ii) The set of integers under addition forms an infinite monoid in which 0 is the 
identity element. 


A non-empty set G together with a binary operation ‘*’ is called a group if for all 
elements a, b, c € G, the following properties hold: 


(i)a*x*bEG (Closure property) 
(ii) a* (b* c) = (a*b) *c (Associative property) 
(iii) de € G such that a *e =e*a=a(Va EG) (Identity Element) 
(iv) For every a € G,da7! € G, such that axa~! = a7! *a=e. (Inverse Element) 


The identity element is unique, and the inverse a _' of an element a is unique (see 
exercise 5). A commutative group has the additional property that a * b = b* a for 
all a, b € G. The order of a finite group G is the number of elements in G, and is 
denoted by o(G). 


Example 6.2 (Groups) 


(i) The set of integers under addition (Z, +) forms an infinite group in which 0 is 
the identity element. 
(ii) The set of 2 x 2 integer matrices under addition, where the identity element 


7a es 
0 0} 
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(iii) |The set of integers under multiplication (Z, x) forms an infinite monoid with 
1 as the identity element. 


A opens group is a group where ca elements g € G are obtained from the 
powers a’ of one element a € G, with a° = e. The element ‘a’ is termed the gen- 
erator of the cyclic group G. A finite cyclic group with n elements is of the form 
ae ge ree ams 

A non-empty subset H of a group G is said iy be a subgroup of G if for all a, 
b € H then a* b € H, and for any a € H thena’' € H. A subgroup N is termed a 
normal subgroup of G if gng | € N for all g € Gand all n € N. Further, if G is a 
group and N is a normal subgroup of G, then the quotient group G/N may be formed. 

Lagrange’s theorem states the relationship between the order of a subgroup H of 
G, and the order of G. The theorem states that if G is a finite group, and His a 
subgroup of G, then o(H) is a divisor of 0(G). 

We may also define a mapping between similar algebraic structures termed 
homomorphism, and these mappings preserve the structure. If the homomorphism is 
one to one and onto, it is termed isomorphism, which means that the two structures 
are identical in some sense (apart from a relabelling of elements). 


6.6.2 Rings 


A ring is a non-empty set R together with two binary operations ‘+’ and ‘x’ where 
(R, +)is acommutative group; (R, x) is asemi-group; and the left and right distributive 
laws hold. Specifically, for all elements a, b, c € R, the following properties hold: 


()a+beR (Closure property) 

(2) a+(b+c) =(a+b)+c (Associative property) 
(3) 30 € R such that Va € R:a+0=0+a=a (Identity element) 

(4) Va € R: H(—a) € R: a+(—a) = (—a)+a=0 (Inverse element) 
(5)a+b=b-+a (Commutativity) 
(6)axbeER (Closure property) 

(7) ax (bx c) =(axb) xc (Associative property) 
(8) ax (b+c) =axb+axc (Distributive law) 

(9) (b+c) xa=bxa+cxa. (Distributive law) 


The element 0 is the identity element under addition, and the additive inverse of 
an element a is given by —a. If a ring (R, x, +) has a multiplicative identity 1 
where a xX | = 1x a=aforalla € R, then R is termed a ring with a unit element. 
Ifaxb=bx a for all a, b € R, then R is termed a commutative ring. 

An element a ¥ 0 in a ring R is said to be a zero divisor if there exists b € R, 
with b ¥ 0 such that ab = 0. A commutative ring is an integral domain if it has no 
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zero divisors. A ring is said to be a division ring if its non-zero elements form a 
group under multiplication. 


Example 6.2 (Rings) 


(i) The set of integers (Z, +, x) forms an infinite commutative ring with mul- 
tiplicative unit element 1. Further, since it has no zero divisors it is an 
integral domain. 

(ii) The set of integers mod 4 (i.e. Z, where addition and multiplication are 
performed modulo 4)! is a finite commutative ring with unit element [1]j,. Its 
elements are {[O]4, [1]a, [2]4, [3]4}. It has zero divisors since [2]4[2]4 = [O]4 
and so it is not an integral domain. 

(iii) The Quaternions (discussed in [1]) are an example of a non-commutative 
ring (they form a division ring). 

(iv) The set of integers mod 5 (i.e. Z; where addition and multiplication are 
performed modulo 5) is a finite commutative division ring” and it has no zero 
divisors. 


6.6.3 Fields 


A field is a non-empty set F together with two binary operation ‘+’ and ‘x’ where 
(F, +) is a commutative group; (F \{0}, x) is a commutative group; and the dis- 
tributive properties hold. The properties of a field are 


()a+beF (Closure property) 

(2) a+(b+c) =(a+b)+c (Associative property) 
(3) 50 € F such that Va € Fa++0=0+a=a (Identity element) 

(4) Va € FA(—a) € Fa+(—a) = (—a)+a=0 (Inverse element) 
(5)a+b=b+a (Commutativity) 
(6)axbeF (Closure property) 

(7) ax (bx c) =(axb) xc (Associative property) 
(8) 41 € F such that Va € Fax 1=1xa=a (Identity element) 

(9) Va € F\ {0}Sa7! € Fax a! =a! xa=1 (Inverse element) 
(10)axb=bxa (Commutativity) 

(11) ax (b+ c) =axb+axc (Distributive law) 
(12) (b+c) xa=bxa+cxa. (Distributive law) 
‘Recall from Chap. 3 that Z/nZ = Z, = {[a],: O<a<n-— 1} = {(O,, [1J,, -.-, [2-1], }- 


?A finite division ring is actually a field (i.e. it is commutative under multiplication), and this 
classic result was proved by Wedderburn. 
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The following are examples of fields: 


Example 6.3 (Fields) 


(i) The set of rational numbers (Q, +, x) forms an infinite commutative field. 
The additive identity is 0, and the multiplicative identity is 1. 

(ii) The set of real numbers (IR, +, x) forms an infinite commutative field. The 
additive identity is 0, and the multiplicative identity is 1. 

(iii) The set of complex numbers (C, +, x) forms an infinite commutative field. 
The additive identity is 0, and the multiplicative identity is 1. 

(iv) The set of integers mod 7 (i.e. Z7 where addition and multiplication are 
performed mod 7) is a finite field. 

(v) The set of integers mod p where p is a prime (i.e. Z, where addition and 
multiplication are performed mod p) is a finite field with p elements. The 
additive identity is [0] and the multiplicative identity is [1]. 


A field is a commutative division ring but not every division ring is a field. For 
example, the quaternions (discovered by Hamilton) are an example of a division 
ring, which is not a field. If the number of elements in the field F is finite then F is 
called a finite field, and F is written as F, where q is the number of elements in F’. In 
fact, every finite field has g = p* elements for some prime p, and some k € N and 
k>0. 


6.6.4 Vector Spaces 


A non-empty set V is said to be a vector space over a field F if V is a commutative 
group under vector addition +, and if for every « € F, v € V there is an element 
ov in V such that the following properties hold for v, w € V and «,f € F: 


u+(v+w) =(u+v)+w. 

50 € V such that Vv € VWW+0=0+4+v=v. 

Yv € VA(—v) € V such that v+(—v) = (—v) +v=0° 
v+w=w-ty. 

a(v-+w) = av+aw. 

(a+ B)v = av+ fy. 

- (Br) = (apy, 


lv=v. 


OAS AAARWN SE 


The elements in V are referred to as vectors and the elements in F are referred to 
as scalars. The element 1 refers to the identity element of the field F under 
multiplication. 
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Application of Vector Spaces to Coding Theory 


The representation of codewords in coding theory (which is discussed in Chap. 11) 
is by n-dimensional vectors over the finite field F,. A codeword vector v is rep- 
resented as the n-tuple: 


v= (ao, a1, ee ini) 


where each a; € Fz. The set of all n-dimensional vectors is the n-dimensional 
vector space F”, with q” elements. The addition of two vectors v and w, where 
V = (0, 41,-.-Gy_1) and w = (bo, bj,...b,_1), is given by 


v+w = (do + bo, a1 +b, ---Gn—1 + Bn-1). 


The scalar multiplication of a vector v = (dg, @,,..-Qn—1) € EF by a scalar B € 
F, is given by 


Bv = (Bao, Bay, ..-Ban-1). 


The set F”, is called the vector space over the finite field F, if the vector space 
properties above hold. A finite set of vectors vj, v2, ... vg is said to be linearly 
independent if 


Byvi + Bova + +++ + Byve =0 B, = Bo =-+ B, =9. 


Otherwise, the set of vectors v1, v2, ... vg is said to be linearly dependent. 

A non-empty subset W of a vector space V(WCV) is said to be a subspace of V, 
if W forms a vector space over F under the operations of V. This is equivalent to 
W being closed under vector addition and scalar multiplication: i.e. w1,w2 € 
W,o«, 8 © F then aw, + Bw2 € W. 

The dimension (dim W) of a subspace WCYV is k if there are k linearly inde- 
pendent vectors in W but every k + | vectors are linearly dependent. A subset of a 
vector space is a basis for V if it consists of linearly independent vectors, and its 
linear span is V (i.e. the basis generates V). We shall employ the basis of the vector 
space of codewords (see Chap. 11) to create the generator matrix to simplify the 
encoding of the information words. The linear span of a set of vectors v1, V2, ..., Vg 
is defined as B,v; + Pyv2 + +++ + Byve. 


Example 6.4 (Vector Spaces) 


(i) The Real coordinate space R” forms an n-dimensional vector space over R. The 
elements of R” are the set of all n-tuples of elements of IR, where an element 
x in R” is written as 
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X = (X1,%2,-.-%n) 


where each x; € R and vector addition and scalar multiplication are given by 


(ii) 


6.7 


OLX = (0X1, 0X2, .. Ky) 


XY = byt, Xo + V2. + Xe +yn)- 


The set of m x n matrices over the real numbers forms a vector space, with 
vector addition given by matrix addition, and the multiplication of a matrix by 
a scalar given by the multiplication of each entry in the matrix by the scalar. 


Review Questions 
. Solve the simple equation: 4(3x + 1) = 7(x+4) — 2(x+5). 
nae ee at) 
. Solve the following simultaneous equations by 
4x — 3y = 18 


(a) Graphical techniques; 
(b) Method of substitution; 
(c) Method of Elimination. 


. Solve the quadratic equation 3x* + 5x — 2 = 0 given that the solution is 


between x = —3 and x = 3 by 


(a) Graphical techniques; 
(b) Factorization; 
(c) Quadratic Formula. 


. Solve the following indicial equation using logarithms 2*=! = 37~!. 
. Explain the differences between semigroups, monoids and groups. 
. Show that the following properties are true for groups: 


(i) The identity element is unique in a group. 
(ii) The inverse of an element is unique in a group. 
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7. Explain the differences between rings, commutative rings, integral 
domains, division rings and fields. 

8. What is a vector space? 

9. Explain how vector spaces may be applied to coding theory (see Chap. 11 
for more details). 


6.8 Summary 


This chapter provided a brief introduction to algebra, which is the branch of 
mathematics that studies mathematical symbols and the rules for manipulating 
them. Algebra is a powerful tool for problem-solving in science and engineering. 

Elementary algebra includes the study of simultaneous equations (i.e. two or 
more equations with two or more unknowns); the solution of quadratic equations 
ax* + bx +c = 0; and the study of polynomials, indices and logarithms. Linear 
algebra is concerned with the solution of a set of linear equations, and the study of 
matrices and vector spaces. 

Abstract algebra is concerned with the study of abstract algebraic structures such 
as monoids, groups, rings, integral domains, fields and vector spaces. The abstract 
approach in modern mathematics has opened up whole new areas of mathematics as 
well as applications in areas such as coding theory in the computing field. 
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Automata Theory 


7.1. Introduction 


Automata Theory is the branch of computer science that is concerned with the study 
of abstract machines and automata. These include finite-state machines, pushdown 
automata and Turing machines. Finite-state machines are abstract machines that 
may be in one of a finite number of states. These machines are in only one state at a 
time (current state), and the input symbol causes a transition from the current state 
to the next state. Finite-state machines have limited computational power due to 
memory and state constraints, but they have been applied to a number of fields 
including communication protocols, neurological systems and linguistics. 

Pushdown automata have greater computational power than finite-state machi- 
nes, and they contain extra memory in the form of a stack from which symbols may 
be pushed or popped. The state transition is determined from the current state of the 
machine, the input symbol and the element on the top of the stack. The action may 
be to change the state and/or push/pop an element from the stack. 

The Turing machine is the most powerful model for computation, and this 
theoretical machine is equivalent to an actual computer in the sense that it can 
compute exactly the same set of functions. The memory of the Turing machine is a 
tape that consists of a potentially infinite number of one-dimensional cells. The 
Turing machine provides a mathematical abstraction of computer execution and 
storage, as well as providing a mathematical definition of an algorithm. However, 
Turing machines are not suitable for programming, and therefore they do not 
provide a good basis for studying programming and programming languages. 
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7.2. Finite-State Machines 


The neurophysiologists Warren McCulloch and Walter Pitts published early work 
on finite-state automata in 1943. They were interested in modelling the thought 
process for humans and machines. Moore and Mealy developed this work further in 
the mid-1950s, and their finite-state machines are referred to as the ‘Mealy machine’ 
and the ‘Moore machine’. The Mealy machine determines its outputs through the 
current state and the input, whereas the output of Moore’s machine is based upon 
the current state alone. 


Definition 7.1 (Finite-State Machine) A finite-state machine (FSM) is an abstract 
mathematical machine that consists of a finite number of states. It includes a start 
state qo in which the machine is in initially; a finite set of states Q; an input alphabet 
&; a state transition function 6 and a set of final accepting states F (where FCQ). 


The state transition function 6 takes the current state and an input symbol, and 
returns the next state. That is, the transition function is of the form: 


6:Qxzr—-Q. 


The transition function provides rules that define the action of the machine for 
each input symbol, and its definition may be extended to provide output as well as a 
transition of the state. State diagrams are used to represent finite-state machines, and 
each state accepts a finite number of inputs. A finite-state machine (Fig. 7.1) may be 
deterministic or non-deterministic, and a deterministic machine changes to exactly 
(or at most)! one state for each input transition, whereas a non-deterministic 
machine may have a choice of states to move for a particular input symbol. 

Finite-state automata can compute only very primitive functions, and so they are 
not adequate as a model for computing. There are more powerful automata such as 
the Turing machine that is essentially a finite automaton with a potentially infinite 
storage (memory). Anything that is computable by a Turing machine. 

A finite-state machine can model a system that has a finite number of states, and 
a finite number of inputs/events that can trigger transitions between states. The 
behaviour of the system at a point in time is determined from the current state and 
input, with behaviour defined for the possible input to that state. The system starts 
in a particular initial state. 

A finite-state machine (also known as finite-state automata) is a quintuple (, Q, 
6, qo, F). The alphabet of the FSM is given by &; the set of states is given by Q; the 
transition function is defined by 6 : Q x X — Q; the initial state is given by go and 
the set of accepting states is given by F where F is a subset of Q. A string is given 
by a sequence of alphabet symbols: i.e. s € &*, and the transition function 5 can be 
extended to 6*: Q x X* — Q. 


'The transition function may be undefined for a particular input symbol and state. 
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Fig. 7.1  Finite-state machine 
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A string s € &* is accepted by the finite-state machine if 6°(go,s) = qr where 
ay © F, and the set of all strings accepted by a finite-state machine is the language 
generated by the machine. A finite-state machine is termed deterministic (Fig. 7.2) 
if the transition function 6 is a function,” and otherwise (where it is a relation) it is 
said to be non-deterministic. A non-deterministic automata is one for which the next 
state is not uniquely determined from the present state and input symbol, and the 
transition may be to a set of states rather than a single state. 

For the example above the input alphabet is given by & = {0,1}; the set of 
states by {A, B, C}; the start state by A; the final state by {C} and the transition 
function is given by the state transition table below (Table 7.1). The language 
accepted by the automata is the set of all binary strings that end with a one that 
contain exactly two ones. 

A non-deterministic automaton (NFA) or non-deterministic finite-state machine 
is a finite-state machine where from each state of the machine and any given input, 
the machine may jump to several possible next states. However, a non-deterministic 
automaton (Fig. 7.3) is equivalent to a deterministic automaton, in that they both 
recognize the same formal language (i.e. regular languages as defined in Chomsky’s 


"It may be a total or a partial function (as discussed in Chap. 2). 
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Fig. 7.3 Non-deterministic finite-state machine 


classification). Further, for any non-deterministic automaton, it is possible to con- 
struct the equivalent deterministic automaton using power set construction. 

NFAs were introduced by Scott and Rabin in 1959, and a NFA is defined 
formally as a 5-tuple (Q, X, 6, qo, F) as in the definition of a deterministic 
automaton, and the only difference is in the transition function 6. 


6:0xx—Pd@. 


The non-deterministic finite-state machine M; = (Q,2,6,qo,F) may be con- 
verted to the equivalent deterministic machine M) = (Q/, x, 6/, q/,, F/) where 


Q' = PO(the set of all subsets of Q) 
I = {Go} 
F'={qe QandqnF #9} 
5'(q,0) = Upeq5(p, @) for each state q € Q' anda € &. 


The set of strings (or language) accepted by an automaton M is denoted L(M). 
That is, L(M) = {s : |0°(qo,s) = q¢ for some gr € F}. A language is termed reg- 
ular if it is accepted by some finite-state machine. Regular sets are closed under 
union, intersection, concatenation, complement and transitive closure. That is, for 
regular sets A, BC* then 


e AUB and ANB are regular. 
e >*\A (ie. A‘) is regular. 
e AB and A* is regular. 


The proof of these properties is demonstrated by constructing finite-state 
machines to accept these languages. For example, the proof that AMB is regular is 
to construct a machine Ma jpg that mimics the execution of M, and Mg and is ina 
final state if and only if both Ma and Mg are in a final state. Finite-state machines 
are useful in designing systems that process sequences of data. 
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Fig. 7.4 Components of pushdown automata 


7.3. Pushdown Automata 


A pushdown automaton (PDA) is essentially a finite-state machine with a stack, and 
it includes three components namely an input tape; a control unit and a potentially 
infinite stack. The stack head scans the top symbol of the stack, and two operations 
(push or pop) may be performed on the stack. The push operation adds a new 
symbol to the top of the stack, whereas the pop operation reads and removes an 
element from the top of the stack (Fig. 7.4). 

A push down automaton may remember a potentially infinite amount of infor- 
mation, whereas a finite-state automaton remembers only a finite amount of 
information. A PDA also differs from a FSM in that it may use the top of the stack 
to decide on which transition to take, and it may manipulate the stack as part of the 
performance of a transition. The input and current state determine the transition in a 
finite-state machine, and a FSM has no stack to work with. 

A pushdown automaton is defined formally as a 7-tuple (2, Q, T, 6, go, Z, F). 
The set & is a finite set which is called the input alphabet; the set Q is a finite set of 
states; I’ is the set of stack symbols; 6 is the transition function which maps 
QO x {= U {e}}* x T into finite subsets of O x T'*?; qo is the initial state; Z is the 
initial stack top symbol on the stack (i.e. Z € I’) and F is the set of accepting states 
(ie. FCQ). 

Figure 7.5 shows a transition from state q, to gz, which is labelled as a, 
b — c. This means that at if the input symbol a occurs in state g,, and the symbol 
on the top of the stack is b, then b is popped from the stack and c is pushed onto the 
stack. The new state is qo. 

In general, a pushdown automaton has several transitions for a given input 
symbol, and so pushdown automata are mainly non-deterministic. If a pushdown 
automaton has at most one transition for the same combination of state, input 


>The use of {ZU {e}} is to formalize that the PDA can either read a letter from the input, or 
proceed leaving the input untouched. 
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Fig. 7.5 Transition in pushdown automata 


symbol and top of stack symbol it is said to be a deterministic PDA (DPDA). The 
set of strings (or language) accepted by a pushdown automaton M is denoted L(M). 

The class of languages accepted by pushdown automata is the context free 
languages, and every context free grammar can be transformed into an equivalent 
non-deterministic pushdown automaton. Chapter 12 has more detailed information 
on the classification of languages. 


Example (Pushdown Automata) 


Construct a non-deterministic pushdown automaton which recognizes the language 
{O"1l" |n > O}. 


Solution 
We construct a pushdown automaton M = (2,Q,T,6,q0,Z,F) where X= 
{0, 1};Q = {q0,91,9 $3 T = {A, Z}; qo is the start state; the start stack symbol is 


Z and the set of accepting states is given by {q;}:. The transition function (relation) 
6 is defined by 


1. aes Z) — (qo,AZ) 
2. (qo,0,A) — (qo, AA) 
3. (40,8,Z) > (41,Z) 
4, (go, 8,4) > (91,4) 
5. (@i,1, A) > (41, €) 
6. (1,8,Z) > (qf,Z) 


The transition function (Fig. 7.6) essentially says that whenever the value 0 
occurs in state gp then A is pushed onto the stack. Parts (3) and (4) of the transition 
function essentially state that the automaton may move from state qo to state q; at 
any moment. Part (5) states when the input symbol is | in state g, then one symbol 
A is popped from the stack. Finally, part (6) states the automaton may move from 
state q, to the accepting state gy only when the stack consists of the single stack 
symbol Z. 
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Fig. 7.6 Transition function for pushdown automata M 


For example, it is easy to see that the string 0011 is accepted by the automaton, 
and the sequence of transitions is given by 


(qo, 0011, Z) (qo,011,AZ) (qo, 11,AAZ) (qi, 11, AAZ) (q1, 1, AZ) (q1,€,Z) (Ge, Z). 


7.4 Turing Machines 


Turing introduced the theoretical Turing Machine in 1936, and this abstract 
mathematical machine consists of a head and a potentially infinite tape that is 
divided into frames. Each frame may be either blank or printed with a symbol from 
a finite alphabet of symbols. The input tape may initially be blank or have a finite 
number of frames containing symbols. At any step, the head can read the contents 
of a frame; the head may erase a symbol on the tape, leave it unchanged or replace it 
with another symbol. It may then move one position to the right, one position to the 
left or not at all. If the frame is blank, the head can either leave the frame blank or 
print one of the symbols (Fig. 7.7). 

Turing believed that a human with finite equipment and with an unlimited 
supply of paper to write on could do every calculation. The unlimited supply of 
paper is formalized in the Turing machine by a paper tape marked off in squares, 
and the tape is potentially infinite in both directions. The tape may be used for 
intermediate calculations as well as for input and output. The finite number of 


Potentially Infinite Tape 


Tape Head (move left or right) 


Transition Function 
Finite Set of States 


Fig. 7.7 Turing machine 
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configurations of the Turing machine was intended to represent the finite states of 
mind of a human calculator. 

The transition function determines for each state and the tape symbol what the 
next state to move to and what should be written on the tape, and where to move the 
tape head. 


Definition 7.2 (Turing Machine) A Turing machine M = (0,1 ,b, 2,6, qo, F) is a 
7-tuple as defined formally in [1] as 


Q is a finite set of states. 
I is a finite set of the tape alphabet/symbols. 

e bé€T is the blank symbol (This is the only symbol that is allowed to occur 
infinitely often on the tape during each step of the computation). 

e > is the set of input symbols and is a subset of I’ (i.e. F = ZU {b}). 

e 6:0xT-OxIx {L, R}* is the transition function. This is a partial func- 
tion where L is left shift and R is right shift. 

© go € Q is the initial state. 

e FCQ is the set of final or accepting states. 


The Turing machine is a simple machine that is equivalent to an actual physical 
computer in the sense that it can compute exactly the same set of functions. It is 
much easier to analyze and prove things about than a real computer, but it is not 
suitable for programming and therefore does not provide a good basis for studying 
programming and programming languages. 

Figure 7.8 illustrates the behaviour when the machine is in state q, and the 
symbol under the tape head is a, where b is written to the tape and the tape head 
moves to the left and the state changes to qo. 

A Turing machine is essentially a finite-state machine (FSM) with an unbounded 
tape. The tape is potentially infinite and unbounded, whereas real computers have a 
large but finite store. The machine may read from and write to the tape. The FSM is 
essentially the control unit of the machine, and the tape is essentially the store. 
However, the store in a real computer may be extended with backing tapes and 
disks, and in a sense may be regarded as unbounded. However, the maximum 
amount of tape that may be read or written within n steps is n. 

A Turing machine has an associated set of rules that defines its behaviour. Its 
actions are defined by the transition function. It may be programmed to solve any 
problem for which there is an algorithm. However, if the problem is unsolvable then 


Fig. 7.8 Transition on turing machine 
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the machine will either stop or compute forever. The solvability of a problem may 
not be determined beforehand. There is, of course, some answer (i.e. either the 
machine halts or it computes forever). The applications of the Turing machine to 
computability and decidability are discussed in Chap. 13. 

Turing also introduced the concept of a Universal Turing Machine, and this 
machine is able to simulate any other Turing machine. 


7.5 Hybrid Automata 


Hybrid systems are digital real-time systems embedded in analog environments 
such as a digital embedded control program for an analog plant environment. The 
controller state moves discretely between the control modes, and within each 
control mode the plant state evolves continuously according to physical laws. That 
is, hybrid systems arise whenever continuous and discrete dynamics interact, and 
where logic decision-making and embedded control actions are combined with 
physical processes. 

This has led to the need for mathematical models to combine the dynamics of the 
continuous part of the system with the dynamics of the logic and discrete parts of 
the system. These models may include some form of differential and difference 
equations for the continuous part on the one hand, and to automata or other discrete 
event models on the other hand. These models assist the challenge of 
multi-disciplinary design, and promote a common understanding of the design 
among the multiple groups of people involved. The physics and mechanics of the 
system are generally controlled by the logic decision making part (i.e. the logical 
device or embedded controller) of the system. Communication becomes more 
challenging in larger systems with complicated networks of communication, and 
this is another dimension of the design that needs to be considered. 

Hybrid automata are a formal model that combine discrete control graphs 
(finite-state automata) with continuously evolving variables. A hybrid automaton 
exhibits two kinds of state changes: discrete jump transitions and continuous flow 
transitions that occur over time. Hybrid systems are often safety critical systems 
where reliability is a central concern, such as the correctness of a digital controller 
that monitors the temperature of a nuclear reactor. We illustrate the idea of hybrid 
automata in the following example. 


Example 


Consider a simple hybrid system for the regulation of temperature in a house, and in 
this simplified description the heating system is assumed to work at maximum 
power or switched off completely. That is, the system can operate in two modes 
namely ‘off’ or ‘on’, and in each mode gq € {off, on} the evolution of the tem- 
perature T can be described by a differential equation (Fig. 7.9). 
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Fig. 7.9 Simple hybrid 
automaton for temperature 
control 
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The system has a hybrid state (q, T) with the state q taking the discrete values 
‘on’ and ‘off’, and a continuous state T taking Real values. The value of the discrete 
state q affects the evolution of the continuous state T, and changes of the discrete 
state g are determined from the continuous state T and conditions on T. The edges 
in the automaton indicate possible discrete state transitions. 


7.6 


Review Questions 


What is a finite-state machine? 

Explain the difference between a deterministic and non-deterministic 
finite-state machine. 

Show how to convert the non-deterministic finite-state automaton in 
Fig. 7.3 to a deterministic automaton. 

What is a pushdown automaton? 

What is a Turing machine? 

Explain what is meant by the language accepted by an automaton. 

Give an example of a language accepted by a pushdown automaton but 
not by a finite-state machine. 

Describe the applications of the Turing machine to computability and 
decidability. 

What is a hybrid automaton? 
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7.7. Summary 


Automata Theory is concerned with the study of abstract machines and automata. 
These include finite-state machines, pushdown automata and Turing machines. 
Finite-state machines are abstract machines that may be in one of a finite number of 
states. These machines are in only one state at a time (current state), and the state 
transition function determines the new state from the current state and the input 
symbol. Finite-state machines have limited computational power due to memory 
and state constraints, but they have been applied to a number of fields including 
communication protocols and linguistics. 

Pushdown automata have greater computational power than finite-state machi- 
nes, and they contain extra memory in the form of a stack from which symbols may 
be pushed or popped. The state transition is determined from the current state of the 
machine, the input symbol and the element on the top of the stack. The action may 
be to change the state and/or push/pop an element from the stack. 

The Turing machine is the most powerful model for computation, and it is 
equivalent to an actual computer in the sense that it can compute exactly the same 
set of functions. The Turing machine provides a mathematical abstraction of 
computer execution and storage, as well as providing a mathematical definition of 
an algorithm. 

Hybrid automata combine finite-state automata with continuously evolving 
variables, and they are often safety critical systems where reliability is a key 
concern. 


Reference 


1. Hopcroft JE, Ullman JD (1979) Introduction to automata theory, languages and computation. 
Addison-Wesley, Boston 


® 


Check for 
updates 


Matrix Theory 


8.1. Introduction 


A matrix is a rectangular array of numbers that consists of horizontal rows and 
vertical columns. A matrix with m rows and n columns is termed an m xX n matrix, 
where m and n are its dimensions. A matrix with an equal number of rows and 
columns (e.g. n rows and n columns) is termed a square matrix. Figure 8.1 is an 
example of a square matrix with four rows and four columns. 

The entry in the ith row and the jth column of a matrix A is denoted by A[i, j], 
Ajj, Or aj, and the matrix A may be denoted by the formula for its (i, j)th entry: i.e. 
(aj) where i ranges from | to m and j ranges from | to n. 

Anm xX | matrix is termed a column vector, and a1 x n matrix is termed a row 
vector. Any row or column of am x n matrix determines a row or column vector 
which is obtained by removing the other rows (respectively columns) from the 
matrix. For example, the row vector (11, —5, 5, 3) is obtained from the matrix 
example by removing rows 1, 2 and 4 of the matrix. 

Two matrices A and B are equal if they are both of the same dimensions, and if 
ay = bi for each i = 1, 2, ..., m and each j = 1, 2, ... n. 

Matrices be added or multiplied (provided certain conditions are satisfied). There 
are identity matrices under the addition and multiplication binary operations such 
that the addition of the (additive) identity matrix to any matrix A yields A and 
similarly for the multiplicative identity. Square matrices have inverses (provided 
that their determinant is non-zero), and every square matrix satisfies its character- 
istic polynomial. 

It is possible to consider matrices with infinite rows and columns, and although it 
is not possible to write down such matrices explicitly it is still possible to add, 
subtract and multiply by a scalar provided there is a well-defined entry in each (i, /) 
th element of the matrix. 

Matrices are an example of an algebraic structure known as an algebra. Chapter 6 
discussed several algebraic structures such as groups, rings, fields and vector spaces. 
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6 0 2 3 
49 39 4 
i 6. 5 3 
3-5 8 1 


Fig. 8.1 Example of a 4 x 4 square matrix 


The matrix algebra for m x n matrices A, B, C and scalars /, pw satisfies the fol- 
lowing properties (there are additional multiplicative properties for square matrices). 


A+B=B+A 
A+(B+C) =(A+B)+C 
A+0=0+A=A 

A+(—A) =(-A)+A=0 
MA+B) =AA+2B 

. (A+py)A = AA+ UB 

» MpA) = (Ap)A 

IA=A. 


SNDAARWNS 


Matrices have many applications including their use in graph theory to keep 
track of the distance between pairs of vertices in the graph; a rotation matrix may be 
employed to represent the rotation of a vector in three-dimensional space. The 
product of two matrices represents the composition of two linear transformations, 
and matrices may be employed to determine the solution to a set of linear equations. 
They also arise in computer graphics and may be employed to project a 
three-dimensional image onto a two-dimensional screen. It is essential to employ 
efficient algorithms for matrix computation, and this is an active area of research in 
the field of numerical analysis. 


8.2. Two x Two Matrices 


Matrices arose in practice as a means of solving a set of linear equations. One of 
the earliest examples of their use is in a Chinese text dating from between 300BC 
and 200AD. The Chinese text showed how matrices could be employed to solve 
simultaneous equations. Consider the set of equations: 


ax+by=r 


cx+dy=s 
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Then, the coefficients of the linear equations in x and y above may be represented 
by the matrix A, where A is given by 


a b 
a=(* 3). 
c d 
The linear equations may be represented as the multiplication of the matrix A 
and a vector x resulting in a vector v: 
Ax =v. 


The matrix representation of the linear equations and its solution are as follows: 


a b x\ f(r 
c d}\y) \s/}r 
The vector x may be calculated by determining the inverse of the matrix A 
(provided that its inverse exists). The vector x is then given by 


1=A-y 


The solution to the set of linear equations is then given by 


x\ fa b /y 
y}) \e d sl} 
The inverse of a matrix A exists if and only if its determinant is non-zero, and if 
this is the case the vector x is given by 


Ge ear) 


The determinant of a 2 x 2 matrix A is given by 


det A = ad — cb. 


The determinant of a 2 x 2 matrix is denoted by 


a b 
c d\ 


A key property of determinants is that 


det(AB) = det(A) det(B). 
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The transpose of a 2 x 2 matrix A (denoted by A’) involves exchanging rows 
and columns, and is given by 
Tf .. ac 
wra(2 5) 


The inverse of the matrix A (denoted by A’) is given by 


Ate 1 d —b 
detA\-c a] 
Further, A: A7' = A’! - A = I where I is the identity matrix of the algebra of 
2 x 2 matrices under multiplication. That is, 


AA '=A'TA= € ). 


0 1 


The addition of two 2 x 2 matrices A and B is given by a matrix whose entries 
are the addition of the individual components of A and B. The addition of two 
matrices is commutative, and we have 


A+B=B+A= ee aa 
c+r d+s 


where A, B are given by 


_(4@ 6 _(P 4 
a=(o a) Be(0 9) 
The identity matrix under addition is given by the matrix whose entries are all 0, 
and it has the property that A+0=0+A=A. 


(0 0) 


The multiplication of two 2 x 2 matrices is given by 


The multiplication of matrices is not commutative: i.e. AB #4 BA. The multi- 
plicative identity matrix I has the property that A- I =I- A = A, and it is given by 
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=(5 9) 


A matrix A may be multiplied by a scalar A, and this yields the matrix 1A where 
each entry in A is multiplied by the scalar 1. That is the entries in the matrix AA are 
aij. 


8.3 Matrix Operations 


More general sets of linear equations may be solved with m x n matrices (i.e. a 
matrix with m rows and n columns) or square n x n matrices. In this section, we 
consider several matrix operations including addition, subtraction, multiplication of 
matrices, scalar multiplication and the transpose of a matrix. 

The addition and subtraction of two matrices A, B is meaningful if and only if A 
and B have the same dimensions: i.e. they are both m x n matrices. In this case, 
A + B is defined by adding the corresponding entries: 


(A+B),= Ay +By. 


The additive identity matrix for the square n x n matrices is denoted by 0, where 
0 is a nm Xn matrix whose entries are zero: i. r;=0 for all i, 7 where 
1<i<nandl <j<n. 

The scalar multiplication of a matrix A by a scalar k is meaningful and the 
resulting matrix kA is given by 


The multiplication of two matrices A and B is meaningful if and only if the 
number of columns of A is equal to the number of rows of B (Fig. 8.2): i.e. A is an 
m Xn matrix and B is a n X p matrix and the resulting matrix AB is a 
m X p matrix. 

Let A = (a) where i ranges from | to m and j ranges from 1 to n, and let 
B = (b,,) where j ranges from | to n and / ranges from 1 to p. Then, AB is given by 
(ci) where i ranges from | to m and / ranges from | to p with c; given by 


n 
Ci = s AixDu- 
k=l 


That is, the entry (cj) is given by multiplying the i" row inA by the 7” column in 
B followed by a summation. Matrix multiplication is not commutative: i.e. 
AB # BA. 


i 
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Fig. 8.2 Multiplication of two matrices 
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The identity matrix I is an x n matrix and the entries are given by r; where 
rij = 1 and rj = O where i ¥ j (Fig. 8.3). A matrix that has non-zero entries only on 
the diagonal is termed a diagonal matrix. A triangular matrix is a square matrix in 
which all the entries above or below the main diagonal are zero. A matrix is an 
upper triangular matrix if all entries below the main diagonal are zero, and lower 
triangular if all of the entries above the main diagonal are zero. Upper triangular 
and lower triangular matrices form a subalgebra of the algebra of square matrices. 

A key property of the identity matrix is that for all n x n matrices A we have 


AI=IA=A 
The inverse of an x n matrix A is a matrix A ! such that 
AAT=ATA=I 


The inverse A” exists if and only if the determinant of A is non-zero. 

The transpose of a matrix A = (a;) involves changing the rows to columns and 
vice versa to form the transpose matrix A’. The result of the operation is that the 
m X n matrix A is converted to the n x m matrix A’ (Fig. 8.4). It is defined by 
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Fig. 8.4 Transpose of a matrix 


(A") = (A) 1 Sj<nand 1 <i<m. 


A matrix is symmetric if it is equal to its transpose: ie. A = A‘. 


8.4 Determinants 


The determinant is a function defined on square matrices and its value is a scalar. 
A key property of determinants is that a matrix is invertible if and only if its 
determinant is non-zero. The determinant of a 2 x 2 matrix is given by 


a b 
c d 


[= aad — be 


The determinant of a 3 x 3 matrix is given by 


= aei+ bfg+cdh — afh — bdi — ceg. 


we QS 
raf 
~ HO 


Cofactors 


Let A be ann x n matrix. For 1 < i,j < n, the (i, j) minor of A is defined to be 
the (n — 1) x (n — 1) matrix obtained by deleting the i-th row and j-th column of 
A (Fig. 8.5). 


140 8 Matrix Theory 


Fig. 8.5 Determining the a \ 
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The shaded row is the ith row, and the shaded column is the jth column. These 
both are deleted from A to form the (i, 7) minor of A, and this isa(m — 1) x (n — 1) 
matrix. 

The (i, 7) cofactor of A is defined to be (-1)'" times the determinant of the 
(i, j) minor. The (i, j) cofactor of A is denoted by K; (A). 

The cofactor matrix Cof A is formed in this way where the (7, j)th element in the 
cofactor matrix is the (i, 7) cofactor of A. 


Definition of Determinant 
The determinant of a matrix is defined as 


det A = S° AjKj. 
j=l 


Another words the determinant of A is determined by taking any row of A and 
multiplying each element by the corresponding cofactor and adding the results. The 
determinant of the product of two matrices is the product of their determinants. 


det(AB) = det A x det B. 


Definition 


The adjugate of A is the n x n matrix Adj(A) whose (i, j) entry is the (, 7) cofactor 
K;; (A) of A. That is, the adjugate of A is the transpose of the cofactor matrix of A. 


Inverse of A 


The inverse of A is determined from the determinant of A and the adjugate of A. 
That is, 


1 1 
-1 +A T 
Av! = 7 AdjA = 5 (Cof A)". 
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A matrix is invertible if and only if its determinant is non-zero: i.e. A is 
invertible if and only if det(A) 4 0. 


Cramer’s Rule 


Cramer’s rule is a theorem that expresses the solution to a system of linear equa- 
tions with several unknowns using the determinant of a matrix. There is a unique 
solution if the determinant of the matrix is non-zero. 

For a system of linear equations of the Ax = v where x and v are n-dimensional 
column vectors, then if det A 4 0 then the unique solution for each x; is 


det U; 
xi= 
det A 


where U; is the matrix obtained from A by replacing the ith column in A by the v- 
column. 


Characteristic Equation 


For every n X n matrix A, there is a polynomial equation of degree n satisfied by 
A. The characteristic polynomial of A is a polynomial in x of degree n. It is given 
by 


cA(x) = det (xI — A). 


Cayley—Hamilton Theorem 


Every matrix A satisfies its characteristic polynomial: i.e. p(A) = 0 where p(x) is the 
characteristic polynomial of A. 


8.5 Eigen Vectors and Values 


A number / is an eigenvalue of an x n matrix A if there is a non-zero vector v such 
that the following equation holds: 


Av =). 
The vector v is termed an eigenvector and the equation is equivalent to 
(A — Aly = 0. 


This means that (A — AI) is a zero divisor and hence it is not an invertible 
matrix. Therefore, 


det(A — Al) = 0. 
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The polynomial function p() = det(A — XI) is called the characteristic poly- 
nomial of A, and it is of degree n. The characteristic equation is p(X) = 0 and as the 
polynomial is of degree n there are at most n roots of the characteristic equation, 
and so there at most n eigenvalues. 

The Cayley—Hamilton theorem states that every matrix satisfies its characteristic 
equation: i.e. the application of the characteristic polynomial to the matrix A yields 
the zero matrix. 


p(A) = 0. 


8.6 Gaussian Elimination 


Gaussian elimination with backward substitution is an important method used in 
solving a set of linear equations. A matrix is used to represent the set of linear 
equations, and Gaussian elimination reduces the matrix to a triangular or reduced 
form, which may then be solved by backward substitution. 

This allows the set of linear equations (E, to E,,) defined below to be solved by 
applying operations to the equations to reduce the matrix to triangular form. This 
reduced form is easier to solve, and it provides exactly the same solution as the 
original set of equations. The set of equations is defined as 


Ey: ayxXy + ayx2 + st) + AinXn = dD, 
Ey: ayX} + GX. + 23+ + AmnXn = dD, 
E,, > Ani xX] ale an2X2 air aaa na AnnXn = by 


Three operations are permitted on the equations, and these operations transform 
the linear system into a reduced form. They are 


(a) Any equation may be multiplied by a non-zero constant. 

(b) An equation E; may be multiplied by a constant and added to another equation 
E;, with the resulting equation replacing E;. 

(c) Equations E; and E; may be transposed with £; replacing E; and vice versa. 


This method for solving a set of linear equations is best illustrated by an 
example, and we consider an example taken from [1]. Then, the solution to a set of 
linear equations with four unknowns may be determined as follows: 


8.6 Gaussian Elimination 143 


Ey: xy+x. +3x4=4 
Ey: 2x, +%»—-x34+%x4 = 1 


E3: 3x, —x.—-x3+2x4 = —3 
Eq: —x,+2x%. + 3x3 -—x4 = 4. 
First, the unknown x, is eliminated from EF, E3 and Ey, and this is done by 
replacing E, with E, — 2E,; replacing E3 with E; — 3£,; and replacing Ey with 
E, + E,, The resulting system is 


Ey: xytx. +3x4=4 


Ey : X2—-X3 5x4 =-—7 
EB : 4x x3 7X4 = 15 
E4: 3x2 + 3x3 +2x4 = 8. 


The next step is then to eliminate x, from F3 and E4. This is done by replacing E3 
with E; — 4E> and replacing FE, with E, + 3E,, The resulting system is now in 
triangular form and the unknown variable may be solved easily by backward 
substitution. That is, we first use equation Ey, to find the solution to x4 and then we 
use equation EF; to find the solution to x3. We then use equations E2 and E1 to find 
the solutions to x» and x;. 


Ey: xy+tx% +3x%4=4 
E,: X_—x3 — 5x4 = —7 
E3: 3x3 + 13x4 = 13 
E4: — 13x4 = -13. 


The usual approach to Gaussian elimination is to do it with an augmented 
matrix. That is, the set of equations is an x n matrix and it is augmented by the 
column vector to form the augmented n x n+ 1 matrix. Gaussian elimination is 
then applied to the matrix to put it into triangular form, and it is then easy to solve 
the unknowns. 

The other common approach to solving a set of linear equation is to employ 
Cramer’s rule, which was discussed in Sect. 8.4. Finally, another possible (but 
computationally expensive) approach to solving the set of linear equations 
Ax =v is to compute the determinant and inverse of A, and to then compute 
x=A™'y, 
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8.7. Business Applications of Matrices 


There are many applications of matrices in business and economics, and the field of 
linear programming often involves solving systems of linear equations. The orga- 
nization structure of many organizations today is matrix orientated rather than the 
traditional functional structure with single managerial responsibility for a functional 
area in the organization. A matrix organization is one in which there is dual or 
multiple managerial accountability and responsibility, and there are generally two 
chains of command, with the first being along traditional functional lines and the 
second being along project or client Fig. 8.6. 

That is, each person in a matrix organization is essentially reporting to two 
managers: their line manager for the functional area that they work in, and the 
project manager for the project that they are assigned to. The project is a temporary 
activity and so this reporting line ceases on project closure, whereas the functional 
line is more permanent (subject to the regular changes following company reor- 
ganizations as part of continuous improvement). 

Another application of matrices in business is that of a decision matrix that 
allows an organization to make decisions objectively based on criteria. For 
example, the tool evaluation matrix in Table 8.1 lists all of the requirements ver- 
tically that the tool is to satisfy, and the candidate tools that are to be evaluated and 
rated against each requirement are listed horizontally. Various rating schemes may 
be employed, and a simple numeric mechanism is employed in the example. The 
tool evaluation criteria are used to rate the effectiveness of each candidate tool, and 


Fig. 8.6 Matrix organization 
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Table 8.1 Tool evaluation Tool 1 Tool 2 a Tool k 
matrix ; 
Requirement | 8 7 9 
Requirement 2 4 6 8 
Requirement n 3 6 8 
Total 35 38 cee 45 


indicate the extent to which the tool satisfies the defined requirements. The chosen 
tool in this example is Tool k as it is the most highly rated of the evaluated tools. 

There are many applications of matrices in the computing field including in 
cryptography, coding theory and graph theory. For more detailed information on 
matrix theory see [2]. 


8.8 Review Questions 


1. Show how 2 x 2 matrices may be added and multiplied 

2. What is the additive identity for 2 x 2 matrices? The multiplicative 
identity? 

3. What is the determinant of a 2 x 2 matrix? 

4. Show that a 2 x 2 matrix is invertible if its determinant is non-zero 

5. Describe general matrix algebra including addition and multiplication, 
determining the determinant and inverse of a matrix 

6. What is Cramer’s rule? 

7. Show how Gaussian elimination may be used to solve a set of linear 
equations 

8. Write a program to find the inverse of a 3 x 3 and then a(n x n) matrix 


8.9 Summary 


A matrix is a rectangular array of numbers that consists of horizontal rows and 
vertical columns. A matrix with m rows and n columns is termed an m X n matrix, 
where m and n are its dimensions. A matrix with an equal number of rows and 
columns (e.g. n rows and n columns) is termed a square matrix. 

Matrices arose in practice as a means of solving a set of linear equations, and one 
of the earliest examples of their use is from a Chinese text dating from between 
300BC and 200AD. 
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Matrices of the same dimensions may be added, subtracted and multiplied by a 
scalar. Two matrices A and B may be multiplied provided that the number of 
columns of A equals the number of rows in B. 

Matrices have an identity matrix under addition and multiplication, and a square 
matrix has an inverse provided that its determinant is non-zero. The inverse of a 
matrix involves determining its determinant, constructing the cofactor matrix and 
transposing the cofactor matrix. 

The solution to a set of linear equations may be determined by Gaussian 
elimination to convert the matrix to upper triangular form, and then employing 
backward substitution. Another approach is to use Cramer’s rule. 

Eigenvalues and eigenvectors lead to the characteristic polynomial and every 
matrix satisfies its characteristic polynomial. The characteristic polynomial is of 
degree n, and a square n Xx n matrix has at most n eigen values. 
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Graph Theory 


9.1. Introduction 


Graph theory is a practical branch of mathematics that deals with the arrangements 
of certain objects known as vertices (or nodes) and the relationships between them. 
It has been applied to practical problems such as the modelling of computer net- 
works, determining the shortest driving route between two cities, the link structure 
of a website, the travelling salesman problem and the four-colour problem.! 

Consider a map of the London underground, which is issued to users of the 
underground transport system in London. Then this map does not represent every 
feature of the city of London, as it includes only material that is relevant to the users 
of the London underground transport system. In this map the exact geographical 
location of the stations is unimportant, and the essential information is how the 
stations are interconnected to one another, as this allows a passenger to plan a route 
from one station to another. That is, the map of the London underground is 
essentially a model of the transport system that shows how the stations are 
interconnected. 

The seven bridges of Kénigsberg” (Fig. 9.1) is one of the earliest problems in 
graph theory. The city was set on both sides of the Pregel River in the early 
eighteenth century, and it consisted of two large islands that were connected to each 


'The 4-colour theorem states that given any map it is possible to colour the regions of the map with 
no more than four colours such that no two adjacent regions have the same colour. This result was 


finally proved in the mid-1970s. 
K6nigsberg was founded in the thirteenth century by Teutonic knights and was one of the cities of 


the Hanseatic League. It was the historical capital of East Prussia (part of Germany), and it was 
annexed by Russia at the end of the Second World War. The German population either fled the 
advancing Red army or were expelled by the Russians in 1949. The city is now called Kaliningrad. 
The famous German philosopher, Immanuel Kant, spent all his life in the city, and is buried there. 


© The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 147 
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River 
Pregel 


Fig. 9.1 K6nigsberg seven bridges problem 


Fig. 9.2 K6nigsberg graph 


other and the mainland by seven bridges. The problem was to find a walk through 
the city that would cross each bridge once and once only. 

Euler showed that the problem had no solution, and his analysis helped to lay the 
foundations for graph theory as a discipline. This problem in graph theory is con- 
cerned with the question as to whether it is possible to travel along the edges of a 
graph starting from a vertex and returning to it and travelling along each edge exactly 
once. An Euler Path in a graph G is a simple path containing every edge of G. 

Euler noted, in effect, that for a walk through a graph traversing each edge 
exactly once depends on the degree of the nodes (i.e. the number of edges touching 
it). He showed that a necessary and sufficient condition for the walk is that the 
graph is connected and has zero or two nodes of odd degree. For the K6ningberg 
graph, the four nodes (i.e. the land masses) have odd degree (Fig. 9.2). 

A graph is a collection of objects that are interconnected in some way. The 
objects are typically represented by vertices (or nodes), and the interconnections 
between them are represented by edges (or lines). We distinguish between directed 
and adirected graphs, where a directed graph is mathematically equivalent to a 
binary relation, and an adirected (undirected) graph is equivalent to a symmetric 
binary relations. 


9.2 Undirected Graphs 


An undirected graph (adirected graph) (Fig. 9.3) G is a pair of finite sets (V, E) 
such that E is a binary symmetric relation on V. The set of vertices (or nodes) is 
denoted by V(G), and the set of edges is denoted by E(G). 
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Fig. 9.3. Undirected graph 
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Fig. 9.4 Directed graph 


A directed graph (Fig. 9.4) is a pair of finite sets (V, E) where E is a binary 
relation (that may not be symmetric) on V. A directed acylic graph (dag) is a 
directed graph that has no cycles. The example below is of a directed graph with 
three edges and four vertices. 

An edge e € E consists of a pair <x, y> where x, y are adjacent nodes in the 
graph. The degree of x is the number of nodes that are adjacent to x. The set of 
edges is denoted by E(G), and the set of vertices is denoted by V(G). 

A weighted graph is a graph G = (V, E) together with a weighting function 
w: E—N, which associates a weight with every edge in the graph. A weighting 
function may be employed in modelling computer networks: for example, the 
weight of an edge may be applied to model the bandwidth of a telecommunications 
link between two nodes. Another application of the weighting function is in 
determining the distance (or shortest path) between two nodes in the graph (where 
such a path exists). 

For an directed graph, the weight of the edge is the same in both directions: i.e. 
w(vi, vj) = w(vj, vi) for all edges <v,, vj> in the graph G, whereas the weights may 
be different for a directed graph. 

Two vertices x, y are adjacent if xy € E, and x and y are said to be incident to the 
edge xy. A matrix may be employed to represent the adjacency relationship. 
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Q 


Example 9.1 Consider the graph G = (V, E) where E = {u = ab, v = cd, w = fe, 
x = bg, y= af}. 


An adjacency matrix (Fig. 9.5) may be employed to represent the relationship of 
adjacency in the graph. Its construction involves listing the vertices in the rows and 
columns, and an entry of | is made in the table if the two vertices are adjacent and 0 
otherwise. 

Similarly, we can construct a table describing the incidence of edges and vertices 
by constructing an incidence matrix (Fig. 9.6). This matrix lists the vertices and 
edges in the rows and columns, and an entry of 1 is made if the vertex is one of the 
nodes of the edge and 0 otherwise. 

Two graphs G = (V, E) and G’ = (V’, E’) are said to be isomorphic if there exists 
a bijection f: V— V’ such that for any u,v € V,uv €E,f(u)f(v) € E’. The 
mapping fis called an isomorphism. Two graphs that are isomorphic are essentially 
equivalent apart from a re-labelling of the nodes and edges. 

Let G = (V, E) and G’ = (V’, E’) be two graphs then G' is a subgraph of G if 
V'CV and E’CE. Given G = (V, E) and V’CV then we can induce a subgraph G 
'=(V’, E’ by restricting G to V’ (denoted by G[/V’|). The set of edges in E’ is 
defined as 


EF’ = {ee E:e = wandu,ve V'}. 


Fig. 9.5 Adjacency matrix abedefeg 
a}/0 1000 1 0 
6}1 000 0 0 1 
e|0 0 01 0 0 0 
d|0 0 10 0 0 0 
e|/0 0 0 0 0 0 0 
fj}1 0 00 0 0 1 
g/0 100 0 1 0 
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Fig. 9.6 Incidence matrix eae We oe 
aj/l 0 00 1 
bj1 0 01 0 
c|0 100 0 
d|0 10 0 0 
e|/0 0 0 0 0 
f jo 0 1 0 1 
g/0 0 1 1 0 


The degree of a vertex v is the number of distinct edges incident to v. It is 
denoted by deg v where 


degv = |{e €E: e = vx for some x € V}} 
= |{x € V: vx € E}|. 


A vertex of degree 0 is called an isolated vertex. 


Theorem 9.1 Let G = (V, E) be a graph then 


LYyeydeg v = 2 |E]. 


Proof This result is clear since each edge contributes one to each of the vertex 
degrees. The formal proof is by induction based on the number of edges in the 
graph, and the basis case is for a graph with no edges (i.e. where every vertex is 
isolated), and the result is immediate for this case. 


The inductive step (strong induction) is to assume that the result is true for all 
graphs with k or fewer edges. We then consider a graph G = (V, E) with k + 1 edges. 

Choose an edge e = xy € E and consider the graph G’ = (V, E’) where E’ = E \ 
{e}. Then, G’ is a graph with k edges and therefore letting deg’ v represent the 
degree of a vertex in G’ we have 


Dev’ v = 2|E'| = 2(\E| — 1) = 2|E| —2. 
The degree of x and y are one less in G’ than they are in G. That is, 


Lev" v-2= See v= 2|E| —2 
339° v= 216). 


A graph G = (V, E) is said to be complete if all the vertices are adjacent: i.e. 
E=V x V.A graph G = (V, E) is said to be simple graph if each edge connects 
two different vertices, and no two edges connect the same pair of vertices. Simi- 
larly, a graph that may have multiple edges between two vertices is termed a 
multigraph. 
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A common problem encountered in graph theory is determining whether or not 
there is a route from one vertex to another. Often, once a route has been identified 
the problem then becomes that of finding the shortest or most efficient route to the 
destination vertex. A graph is said to be connected if for any two given vertices vj, 
v2 in V there is a path from v, to vp. 

Consider a person walking in a forest from A to B where the person does not 
know the way to B. Often, the route taken will involve the person wandering 
around aimlessly, and often retracing parts of the route until eventually the desti- 
nation B is reached. This is an example of a walk from v, to v; where there may be 
repetition of edges. 

If all of the edges of a walk are distinct then it is called a trail. A path vy, Vo, ..., 
v, from vertex v, to v, is of length k — 1 and consists of the sequence of edges <v,, 
V2>, <V2, V3>, ..., <Ve1, Ve> Where each <v;, v;,;> is an edge in E. The vertices in the 
path are all distinct apart from possibly v,; and v,. The path is said to be a cycle if 
Vv, = Vy. A graph is said to be acyclic if it contains no cycles. 


Theorem 9.2 Let G = (V, E) be a graph and W = vj, V2, ..., Vk be a walk from v, 
to vy. Then there is a path from v; to vx, using only edges of W. 

Proof The walk W may be reduced to a path by successively replacing redundant 
parts in the walk of the form v; v4), ..., vj where v; =v, with v; That is, we 
successively remove cycles from the walk and this clearly leads to a path (not 
necessarily the shortest path) from v, to vx. 

Theorem 9.3 Let G = (V, E) be a graph and let u, v © Vwith u 4 v. Suppose that 


there exists two different paths from u to v in G, then G contains a cycle. 


Wrt1 W,. 


Suppose that P = vy, vo, ..., v, and Q = wy, Wo, ..., Wm are two distinct paths 
from u to v (where u # v), and u = vl = wl and v = vn = wm. Suppose P and Q 
are identical for the first k vertices (kK could be 1), and then differ (i.e. vk + 1 4 
wk + 1). Then Q crosses P again at v,, = w,,, and possibly several times before then. 
Suppose the first occurrence is at vj=w, with k<i<n. Then 
Why Wk +15 Wk42s+++)WjVi-1, Vi-2,+++, Ve iS a closed path (i.e. a cycle) since the 
vertices are all distinct. 

If there is a path from vj, to v2 then it is possible to define the distance between v, 
and v,. This is defined to be the total length (number of edges) of the shortest path 
between v, and v5. 
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A Hamiltonian path* in a graph G = (V, E) is a path that visits every vertex once 
and once only. Another words, the length of a Hamiltonian path is |V| — 1. A graph 
is Hamiltonian-connected if for every pair of vertices there is a Hamiltonian path 
between the two vertices. 

Hamiltonian paths are applicable to the travelling salesman problem, where a 
salesman‘ wishes to travel to k cities in the country without visiting any city more 
than once. In principle, this problem may be solved by looking at all of the possible 
routes between the various cities, and choosing the route with the minimal distance. 

For example, Fig. 9.7 shows five cities and the connections (including distance) 
between them. Then, a travelling salesman starting at A would visit the cities in the 
order AEDCBA (or in reverse order ABCDEA) covering a total distance of 14. 

However, the problem becomes much more difficult to solve as the number of 
cities increases, and there is no general algorithm for its solution. For example, for 
the case of ten cities, the total number of possible routes is given by 9! = 362,880, 
and an exhaustive search by a computer is feasible and the solution may be 
determined quite quickly. However, for 20 cities, the total number of routes is given 
by 19! = 1.2 x 10'’, and in this case it is no longer feasible to do an exhaustive 
search by a computer. 

There are several sufficient conditions for the existence of a Hamiltonian path, 
and Theorem 9.4 describes a condition that is sufficient for its existence. 


Theorem 9.4 Let G = (V, E) be a graph with |V| =n and such that deg v + deg 
w > n-—1 for all non-adjacent vertices v and w. Then G possesses a Hamiltonian 
path. 

Proof The first part of the proof involves showing that G is connected, and the 
second part involves considering the largest path in G of length k — 1 and assuming 
that k <n. A contradiction is then derived and it is deduced that k = n. 


We assume that G’ = (V’, E’) and G” = (V"", E”) are two connected components 
of G, then |V’|+|V”|<n and so if ve Vv’ and weV" then 
n—-1<°v+ows|V']—14+]/V"] —1=]VW]+]V"| —2<n—2 which is a con- 
tradiction, and so G must be connected. 

Let P = vy, vo, ..., ve be the largest path in G and suppose k <n. From this a 
contradiction is derived, and the details for are in [1]. 


These are named after Sir William Rowan Hamilton, a nineteenth-century Irish mathematician 
and astronomer, who is famous for discovering quaternions [3]. 

4 

We use the term ‘salesman’ to stand for ‘salesman’ or ‘saleswoman’. 
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Fig. 9.7 Travelling salesman problem 


9.3. Trees 


An acylic graph is termed a forest and a connected forest is termed a tree. A graph Gis 
a tree if and only if for each pair of vertices in G there exists a unique path in G joining 
these vertices. This is since G is connected and acyclic, with the connected property 
giving the existence of at least one path and the acylic property giving uniqueness. 

A spanning tree T = (V, E’) for the connected graph G = (V, E) is a tree with the 
same vertex set V. It is formed from the graph by removing edges from it until it is 
acyclic (while ensuring that the graph remains connected). 


Theorem 9.5 Let G = (V, E) be a tree and let e € E then G'=(V, E \fe}) is 
disconnected and has two components. 

Proof Let e = wv then since G is connected and acyclic uv is the unique path from 
u to v, and thus G’ is disconnected since there is no path from u to v in G’. 


It is thus clear that there are at least two components in G’ with u and v in different 
components. We show that any other vertex w is connected to u or to v in G’. 

Since G is connected there is a path from w to u in G, and if this path does not 
use e then it is in G’ as well, and therefore u and w are in the same component of G’. 

If it does use e then e is the last edge of the graph since u cannot appear twice in 
the path, and so the path is of the form w, ..., v, u in G. Therefore, there is a path 
from w to v in G’, and so w and v are in the same component in G’. Therefore, there 
are only two components in G’. 


Theorem 9.6 Any connected graph G = (V, E) possesses a spanning tree. 
Proof This result is proved by considering all connected subgraphs of (G = V, E) 
and choosing a subgraph T with |E'| as small as possible. The final step is to show 
that T is the desired spanning tree, and this involves showing that T is acyclic. The 
details of the proof are left to the reader. 

Theorem 9.7 Let G = (V, E) be a connected graph, then G is a tree if and only if | 
E| = |v - 7. 

Proof This result may be proved by induction on the number of vertices |V| and the 
applications of Theorems 9.5 and 9.6. 
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Fig. 9.8 Binary tree (A) 


9.3.1 Binary Trees 


A binary tree (Fig. 9.8) is a tree in which each node has at most two child nodes 
(termed left and right child nodes). A node with children is termed a parent node, 
and the top node of the tree is termed the root node. Any node in the tree can be 
reached by starting from the root node, and by repeatedly taking either the left 
branch (left child) or right branch (right child) until the node is reached. Binary 
trees are used in computing to implement efficient searching algorithms (We gave 
an alternative recursive definition of a binary tree in Chap. 4). 

The depth of a node is the length of the path (i.e. the number of edges) from the 
root to the node. The depth of a tree is the length of the path from the root to the 
deepest node in the tree. A balanced binary tree is a binary tree in which the depth 
of the two subtrees of any node never differs by more than one. The root of the 
binary tree in Fig. 9.8 is A and its depth is 4. The tree is unbalanced and unsorted. 

Tree traversal is a systematic way of visiting each node in the tree exactly once, 
and we distinguish between breadth first search in which every node on a particular 
level is visited before going to a lower level, and depth first search where one starts 
at the root and explores as far as possible along each branch before backtracking. 
The traversal in depth first search may be in preorder, inorder or postorder. 


9.4 Graph Algorithms 


Graph algorithms are employed to solve various problems in graph theory including 
network cost minimization problems; construction of spanning trees; shortest path 
algorithms; longest path algorithms and timetable construction problems. 

A length function /: E — R may be defined on the edges of a connected graph 
G = (V, B), and a shortest path from u to v in G is a path P with edge set E’ such 
that /(E’) is minimal. 

The reader should consult the many texts on graph theory to explore many 
well-known graph algorithms such as Dijkstra’s shortest path algorithm and longest 
path algorithm (e.g. as described in [1]). Kruskal’s minimal spanning tree algorithm 
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and Prim’s minimal spanning tree algorithms are described in [1]. We briefly 
discuss graph colouring in the next section, but it is not possible due to space 
constraints to describe graph algorithms in more detail in this text. 


9.5 Graph Colouring and Four-Colour Problem 


It is very common for maps to be coloured in such a way that neighbouring states or 
countries are coloured differently. This allows different states or countries to be 
easily distinguished as well as the borders between them. The question naturally 
arises as to how many colours are needed (or determining the least number of 
colours needed) to colour the entire map, as it might be expected that a large 
number of colours would be needed to colour a large complicated map. 

However, it may come as a surprise that in fact very few colours are required to 
colour any map. A former student of the British logician, Augustus De Morgan, had 
noticed this in the mid-1800s, and he proposed the conjecture of the four-colour 
theorem. There were various attempts to prove that 4 colours were sufficient from 
the mid-1800s onwards, and it remained a famous unsolved problem in mathe- 
matics until the late twentieth century. 

Kempe gave an erroneous proof of the four-colour problem in 1879, but his 
attempt led to the proof that five colours are sufficient (which was proved by 
Heawod in the late 1800s). Appel and Haken of the University of Illinois finally 
provided the proof that 4 colours are sufficient in the mid-1970s (using over 1000 h 
of computer time in their proof). 

Each map in the plane can be represented by a graph, with each region of the 
graph represented by a vertex. Edges connect two vertices if the regions have a 
common border. The colouring of a graph is the assignment of a colour to each 
vertex of the graph so that no two adjacent vertices in this graph have the same 
colour. 


Definition 


Let G = (V, E) be a graph and let C be a finite set called the colours. Then, a 
colouring of G is a mapping « : V > C such that if wv € E then x(u) £ K(v). 

That is, the colouring of a simple graph is the assignment of a colour to each 
vertex of the graph such that if two vertices are adjacent then they are assigned a 
different colour. The chromatic number of a graph is the least number of colours 
needed for a colouring of the graph. It is denoted by 7(G). 


Example 9.2. Show that the chromatic colour of the following graph G (Fig. 9.9) is 
3 (this example is adapted from [2]). 
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Fig. 9.9 Determining the 
chromatic colour of G 


Fig. 9.10 Chromatic 
colouring of G 


Solution 


The chromatic colour of G must be at least 3 since vertices p, g and r must have 
different colours, and so we need to show that 3 colours are in fact sufficient to 
colour G. We assign the colours red, blue and green to p, g and r respectively. We 
immediately deduce that the colour of s must be red (as adjacent to g and r). From 
this, we deduce that ¢ is coloured green (as adjacent to g and s) and u is coloured 
blue (as adjacent to s and #). Finally, v must be coloured red (as adjacent to u and £). 
This leads to the colouring of the graph G in Fig. 9.10. 


Theorem 9.8 (Four-Colour Theorem) The chromatic number of a planar graph G 
is less than or equal to 4. 


9.6 Review Questions 


1. What is a graph and explain the difference between an adirected graph 
and a directed graph. 

2. Determine the adjacency and incidence matrices of the following graph 
where V = {a, b, c, d, e} and E = {ab, bc, ae, cd, bd}. 

3. Determine if the two graphs G and G’ defined below are isomorphic. 
G = (V,E), V = {a,b,c,d,e,f,g} andE = {ab, ad, ae, bd, ce, cf, dg, fg, bf}. 
G' = (V', E), V’ = {a, b,c, d, e, f, g} and E’ = {ab, be, cd, de, ef, fg, ga, 
ac, be}. 

4. What is a binary tree? Describe applications of binary trees. 
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Describe the travelling salesman problem and its applications. 
Explain the difference between a walk, trail and path. 
What is a connected graph? 
Explain the difference between an incidence matrix and an adjacency 
matrix. 
9. Complete the details of Theorems 9.6 and 9.7. 
10. Describe the four-colour problem and its applications. 


Co SI ON 


9.7. Summary 


This chapter provided a brief introduction to graph theory, which is a practical 
branch of mathematics that deals with the arrangements of vertices and the edges 
between them. It has been applied to practical problems such as the modelling of 
computer networks, determining the shortest driving route between two cities and 
the travelling salesman problem. 

The seven bridges of K6nigsberg is one of the earliest problems in graph theory, 
and it was concerned with the problem was of finding a walk through the city that 
would cross each bridge once and once only. Euler showed that the problem had no 
solution, and his analysis helped to lay the foundations for graph theory. 

An undirected graph G is a pair of finite sets (V, E) such that E is a binary 
symmetric relation on V, whereas a directed graph is a binary relation that is not 
symmetric. An adjacency matrix is used to represent whether two vertices are 
adjacent to each other, whereas an incidence matrix indicates whether a vertex is 
part of a particular edge. 

A Hamiltonian path in a graph is a path that visits every vertex once and once 
only. Hamiltonian paths are applicable to the travelling salesman problem, where a 
salesman wishes to travel to k cities in the country without visiting any city more 
than once. A tree is a connected and acylic graph, and a binary tree is a tree in 
which each node has at most two child nodes. 

Graph colouring arose to answer the question as to how many colours are needed 
to colour an entire map. It may be expected that many colours would be required, 
but the four-colour theorem demonstrates that in fact four colours are sufficient to 
colour a planar graph. 
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10.1 Introduction 


Cryptography was originally employed to protect communication of private 
information between individuals. Today, it consists of mathematical techniques that 
provide secrecy in the transmission of messages between computers, and its 
objective is to solve security problems such as privacy and authentication over a 
communications channel. 

It involves enciphering and deciphering messages, and it employs theoretical 
results from number theory to convert the original message (or plaintext) into cipher 
text that is then transmitted over a secure channel to the intended recipient. The 
cipher text is meaningless to anyone other than the intended recipient, and the 
recipient uses a key to decrypt the received cipher text and to read the original 
message. 

The origin of the word “cryptography” is from the Greek ‘kryptos’ meaning 
hidden, and ‘graphein’ meaning to write. The field of cryptography is concerned 
with techniques by which information may be concealed in cipher texts and made 
unintelligible to all but the intended recipient. This ensures the privacy of the 
information sent, as any information intercepted will be meaningless to anyone 
other than the recipient. 

Julius Caesar developed one of the earliest ciphers on his military campaigns in 
Gaul. His objective was to communicate important messages safely to his generals. 
His solution is one of the simplest and widely known encryption techniques, and it 
involves the substitution of each letter in the plaintext (i.e., the original message) by 
a letter a fixed number of positions down in the alphabet. The Caesar cipher 
involves a shift of 3 positions and this leads to the letter B being replaced by E, the 
letter C by F, and so on. 

The Caesar cipher (Fig. 10.1) is easily broken, as the frequency distribution of 
letters may be employed to determine the mapping. However, the Gaulish tribes 
who were mainly illiterate, and it is likely that the cipher provided good security. 
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Fig. 10.1 Caesar cipher 


The translation of the Roman letters by the Caesar cipher (with a shift key of 3) can 
be seen by the following table. 

The process of enciphering a message (i.e., the plaintext) simply involves going 
through each letter in the plaintext and writing down the corresponding cipher 
letter. The enciphering of the plaintext message “summer solstice” involves the 
following: 


Plaintext: Summer solstice 


Cipher text vxpphu vrovwleh 


The process of deciphering a cipher message involves doing the reverse oper- 
ation: i.e., for each cipher letter the corresponding plaintext letter is identified from 
the table. 


Cipher text vxpphu vrovwleh 


Plaintext: summer solstice 


The encryption may also be represented using modular arithmetic. This involves 
using the numbers 0-25 to represent the alphabet letters, and the encryption of a 
letter is given by a shift transformation of three (modulo 26). This is simply 
addition (modula 26): i.e., the encloding of the plaintext letter x is given by: 


x+3(mod 26) =a 
Similarly, the decoding of the cipher letter a is given by: 
a — 3(mod 26) = x 


The Caesar cipher was still in use up to the early twentienth century. However, 
by then frequency analysis techniques were available to break the cipher. The 
Vignére cipher uses a Caesar cipher with a different shift at each position in the text. 
The value of the shift to be employed with each plaintext letter is defined using a 
repeating keyword. 
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Fig. 10.2 The enigma 
machine 


10.2 Breaking the Enigma Codes 


The Enigma codes were used by the Germans during the second world war for the 
secure transmission of naval messages to their submarines. These messages con- 
tained top-secret information on German submarine and naval activities in the 
Atlantic, and the threat that they posed to British and Allied shipping. 

The codes allowed messages to be passed secretly using encryption, and this 
meant that any unauthorised inteception was meaningless to the Allies. The 
plaintext (i.e., the original message) was converted by the Enigma machine 
(Fig. 10.2) into the encrypted text, and these messages were then transmitted by the 
German military to their submarines in the Atlantic, or to their bases throughout 
Europe. 

The Enigma machine was invented in 1918 and generated a polyalphabetic 
substitution cipher which the Germans believed to be unbreakable. A letter was 
typed in German into the machine, and electrical impulses through a series of 
rotating wheels and wires produced the encrypted letter which was lit up on a panel 
above the keyboard. The recipient typed the received message into his machine and 
the decrypted message was lit up letter by letter above the keyboard. The rotors and 
wires of the machine could be configured in many different ways, and during the 
war the cipher settings were changed at least once a day. The odds against anyone 
breaking the Enigma machine without knowing the setting were 150 x 10 46:1, 

The British code and cipher school was relocated from London to Bletchley Park 
(Fig. 10.3) at the start of the second world war. It was located is in the town of 
Bletchley near Milton Keynes (about fifty miles north west of London). It was 
commanded by Alistair Dennison and was known as Station X, and several thou- 
sands were working there during the second world war. The team at Bletchley Park 
broke the Enigma codes, and therefore made vital contributions to the British and 
Allied war effort. 
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Fig. 10.3 Bletchley park 


Fig. 10.4 Alan turing 


Polish cryptanalysts did important work in breaking the Enigma machine in the 
early 1930s, and they constructed a replica of the machine. They passed their 
knowledge on to the British and gave them the replica just prior to the German 
invasion of Poland. The team at Bletchley built upon the Polish work, and the team 
included Alan Turing’ (Fig. 10.4) and other mathematicians. 

The code-breaking teams worked in various huts in Bletchley park. Hut 6 
focused on air force and army cyphers, and hut 8 focused on naval cyphers. The 
deciphered messages were then converted into intelligence reports, with air force 
and army intelligence reports produced by the team in hut 3, and naval intelligence 
reports produced by the team in hut 4. The raw material (i.e., the encrpted mes- 
sages) to be deciphered came from wireless intercept stations dotted around Britain, 
and from various countries overseas. These stations listened to German radio 
messages, and sent them to Bletchley park to be deciphered and analysed. 

Turing devised a machine to assist with breaking the codes (an idea that was 
originally proposed by the Polish cryptanalysts). This electromechanical machine 
was known as the bombe (Fig. 10.5) , and its goal was to find the right settings of 


'Turing made fundamental contributions to computing, including the theoretical Turing machine. 
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Fig. 10.5 Replica of bombe 


the Enigma machine for that particular day. The machine greatly reduced the odds 
and the time required to determine the settings on the Enigma machine, and it 
became the main tool for reading the Enigma traffic during the war. The bombe was 
first installed in early 1940 and it weighed over a ton. It was named after a cryp- 
tological device designed in 1938 by the Polish cryptologist, Marian Rejewski. 

A standard Enigma machine employed a set of rotors, and each rotor could be in 
any of 26 positions. The bombe tried each possible rotor position and applied a test. 
The test eliminated almost all of the positions and left a smaller number of cases to 
be dealt with. The test required the cryptologist to have a suitable “crib”: i.e., a 
section of ciphertext for which he could guess the corresponding plaintext. 

For each possible setting of the rotors, the bombe employed the crib to perform a 
chain of logical deductions. The bombe detected when a contradiction had occurred 
and it then ruled out that setting and moved onto the next. Most of the possible 
settings would lead to contradictions and could then be discarded. This would leave 
only a few settings to be investigated in detail. 

The Government Communication Headquarters (GCHQ) was the successor of 
Bletchley Park, and it opened after the war. The site at Bletchley park was then used 
for training purposes. 

The codebreakers who worked at Bletchley Park were required to remain silent 
about their achievements until the mid-1970s when the wartime information was 
declassified.” The link between British Intelligence and Bletchley Park came to an 
end in the mid-1980s. 

It was decided in the mid-1990s to restore Bletchley Park, and today it is run as a 
museum by the Bletchley Park Trust. 


?Gordan Welchman was the head of Hut 6 at Bletchey Park and he published his book ‘The Hut 
Six Story’ in 1982 (in the US and UK). However, the security services disapproved of its 
publication and his security clearance was revoked. He was forbidden to speak of his book and his 
wartime work. 
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Table. 10.1 Notation in Symbol Description 


ceypioeraphy M Represents the message (plaintext) 
ic Represents the encrypted message (cipher text) 
ex Represents the encryption key 
di, Represents the decryption key 
E Represents the encryption process 
D Represents the decryption process 


10.3 Cryptographic Systems 


A cryptographic system is a computer system that is concerned with the secure 
transmission of messages. The message is encrypted prior to its transmission, which 
ensures that any unauthorized interception and viewing of the message is mean- 
ingless to anyone other than the intended recipient. The recipient uses a key to 
decrypt the cipher text, and to retrieve the original message. 

There are essentially two different types of cryptographic systems employed, and 
these are public key cryptosystems and secret key cryptosystems. A public key 
cryptosystem is an asymmetric cryptosystem where two different keys are 
employed: one for encryption and one for decryption. The fact that a person is able 
to encrypt a message does not mean that the person is able to decrypt a message. 

In a secret key cryptosystem the same key is used for both encryption and 
decryption. Anyone who has knowledge on how to encrypt messages has sufficient 
knowledge to decrypt messages. The following notation is employed (Table 10.1). 

The encryption and decryption algorithms satisfy the following equation: 


Dak (C) = Dd (Eex(M)) = M 


There are two different keys employed in a public key cryptosystem. These are 
the encryption key e, and the decryption key d, with e,. 4 d,. It is called asym- 
metric since the encryption key differs from the decryption key. 

There is just one key employed in a secret key cryptosystem, with the same key 
ex. is used for both encryption and decryption. It is called symmetric since the 
encryption key is the same as the decryption key: i.e., ey. = dy. 


10.4 Symmetric Key Systems 


A symmetric key cryptosystem (Fig. 10.6) uses the same secret key for encryption 
and decryption. The sender and the receiver first need to agree a shared key prior to 
communication. This needs to be done over a secure channel to ensure that the 
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Public Channel Hostile Attack 
(Insecure) (Enemy) 
> 


Encryption Decryption Message 
C=E,(M) M= D(C) M 


t t 


i Secure Channel 


Secret Key 
(k) 


Fig. 10.6 Symmetric key cryptosystem 


shared key remains secret. Once this has been done they can begin to encrypt and 
decrypt messages using the secret key. Anyone who is able to encrypt a message 
has sufficient information to decrypt the message. 

The encryption of a message is in effect a transformation from the space of 
messages .% to the space of cryptosystems C. That is, the encryption of a message 
with key k is an invertible transformation f such that: 


f:M*C 


The cipher text is given by C = E,(M) where M € .@ and C € C. The legit- 
imate receiver of the message knows the secret key k (as it will have transmitted 
previously over a secure channel), and so the cipher text C can be decrypted by the 
inverse transformation f ' defined by: 


fF sca “4 


Therefore, we have that D,(C) = D;(E,(M)) =M the original plaintext 
message. 

There are advantages and disadvantages to symmetric key systems (Table 10.2), 
and these include: 


Examples of Symmetric Key Systems 


(i) Caesar Cipher 


The Caesar cipher may be defined using modular arithmetic. It involves a shift of 
three places for each letter in the plaintext, and the alphabetic letters are represented 
by the numbers 0-25. The encyption is carried out by addition (modula 26). The 
encryption of a plaintext letter x to a cipher letter c is given by”: 


3Here x and c are variables rather than the alphabetic characters ‘x’ and ‘c’. 
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Table. 10.2 Advantages and disadvantages of symmetric key systems 


Advantages Disadvantages 


Encryption process is simple (as the same A shared key must be agreed between two 

key is used for encryption and decryption) _ parties 

It is faster than public key systems Key exchange is difficult as there needs to be a 
secure channel between the two parties (to 
ensure that the key remains secret) 


It uses less computer resources than public _I[f a user has n trading partners then n secret 


key systems keys must be maintained (one for each partner) 
It uses a different key for communication There are problems with the management and 
with every different party security of all of these keys (due to volume of 


keys that need to be maintained) 


Authenticity of origin or receipt cannot be 
proved (as key is shared) 


c =x+3(mod 26) 
Similarly, the decryption of a cipher letter c is given by: 


x = c — 3(mod 26) 


(li) Generalized Caesar Cipher 


This is a generalisation of the Caesar cipher to a shift of k (the Caesar cipher 
involves a shift of three). This is given by: 


fie = Ex(x) =x+k(mod26) 0<k<25 


f, | =Dy(c) = c—k(mod 26) 0<k<25 


(iii) Affine Transformation 
This is a more general transformation and is defined by: 
fia) = E(ap)(x) = ax + b(mod 26) O<a,b,x <25 and ged(a, 26) = 1 


Faas) = Dia») (c) =a | (c—b)(mod 26) a7! is the inverse of a mod 26 
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(iv) Block Ciphers 


Stream ciphers encrypt a single letter at a time and are easy to break. Block 
ciphers offer greater security, and the plaintext is split into groups of letters, and the 
encryption is performed on the block of letters rather than on a single letter. 

The message is split into blocks of n-letters: M;, Mo, ... M, where each M; 
(1 < i < k)isa block n letters. The letters in the message are translated into their 
numerical equivalents, and the cipher text formed as follows: 


C; =AM;+B (modN) i=1,2,...k 


where (A, B) is the key, A is an invertible n x n matrix with gcd(det(A), N) = Be 
M; = (m1, ma, ... Wn)", B= (By, ba, ... By)", Cy = (Cs C25 «nes Cn) 


a1 42 443O«w se An my by C1 

a2, 422, 493s» Ay mM by C2 

43, 432, 433,— «sy m3 by | | & 
a = 

ani Gn2  An3 s+ Ann My by Cn 


The decryption is performed by: 
M; =A7'(C;—B)(modN) i=1,2,...k 


-1 


my ay a2 43. CO«w ee An cy — by 
m2 az, 422, 423. «wn c2 — bo 
m3 | | a3 a32 433... ~G3n c3 — b3 
My ani Gn2  a4n3 s+» Ann Ch — by 


(v) Exponential Ciphers 


Pohlig and Hellman [1] invented the exponential cipher in 1976. This cipher is 
less vulnerable to frequency analysis than block ciphers. 

Let p be a prime number and let M be the numerical representation of the 
plaintext, with each letter of the plaintext replaced with its two-digit representation 
(00-25). That is, A = 00, B = O1, ...., Z = 25. 


“This requirement is to ensure that the matrix A is invertible. 
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M is divided into blocks M; (these are equal size blocks of m letters where the 
block size is approximately the same number of digits as p). The number of letters 
m per block is chosen such that: 


2525.....25 <p< 2525.....25 
—~S_——— —_~Y_— 


mtimes m-+ | times 


For example, for the prime 8191 a block size of m = 2 letters (4 digits) is chosen 
since: 


2525 < 8191 <252525 


The secret encryption key is chosen to be an integer k such that 0 < k < p and 
gcd(k, p — 1) = 1. Then the encryption of the block M,; is defined by: 


C; = Ex(M;) = M‘(mod p) 


The cipher text C; is an integer such that0 < C; <p. 

The decryption of C; involves first determining the inverse k | of the key k (mod 
p —1), i.e., we determine k' such that k k*=1 (mod p — 1). The secret key k was 
chosen so that (k, p — 1) = 1, and this means that there are integers d and n such that 
kd = 1+n(p—1), and so k! is d and kk! = 1+n(p — 1). Therefore, 


Dy1(C;) = CE" = (MY)! = M}*"?-) = Mj(modp) 


The fact that M; tap!) — M; (mod p) follows from Euler’s Theorem and 
Fermat’s Little Theorem (Theorem 3.7 and 3.8), which were discussed in Chap. 3. 
Euler’s Theorem states that for two positive integers a and n with gcd(a, n) = 1 that 
a?) =1 (mod n). 

Clearly, for a prime p we have that (p) = p — 1. This allows us to deduce that: 


n 


Mi +"0-) = wim"?-) = M, (me?) = M,(1)"= M,(modp) 


(vi) Data Encryption Standard (DES) 


DES is a popular cryptographic system [2] used by governments and private 
companies around the world. It is based on a symmetric key algorithm and uses a 
shared secret key that is known only to the sender and receiver. It was designed by 
IBM and approved by the National Bureau of Standards (NBS?) in 1976. It is a 
block cipher and a message is split into 64-bit message blocks. The algorithm is 
employed in reverse to decrypt each cipher text block. 


°The NBS is now known as the National Institute of Standards and Technology (NIST). 
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Today, DES is considered to be insecure for many applications as its key size 
(56 bits) is viewed as being too small, and the cipher has been broken in less than 
24h. This has led to it being withdrawn as a standard and replaced by the 
Advanced Encryption Standard (AES), which uses a larger key of 128 bits or 256 
bits. 

The DES algorithm uses the same secret 56-bit key for encryption and 
decryption. The key consists of 56 bits taken from a 64-bit key that includes 8 
parity bits. The parity bits are at position 8, 16, ..., 64, and so every 8" bit of the 
64-bit key is discarded leaving behind only the 56-bit key. 

The algorithm is then applied to each 64-bit message block and the plaintext 
message block is converted into a 64-bit cipher text block. An initial permutation is 
first applied to M to create M’, and M’ is divided into a 32-bit left half Lg and a 
32-bit right half Ro, There are then 16 iterations, with the iterations having a left half 
and a right half: 


L; = Ry-1 
R; = Li-} ® f (Ri-1, Ki) 


The function fis a function that takes a 32 bit right half and a 48 bit round key K; 
(each K; contains a different subset of the 56-bit key) and produces a 32-bit output. 
Finally, the pre-cipher text (Rj6, Li¢) is permuted to yield the final cipher text C. 
The function f operates on half a message block (Table 10.3) and involves: 

The decryption of the cipher text is similar to the encryption and it involves running 
the algorithm in reverse. DES has been implemented on a microchip. However, it has 
been superseded in recent years by AES due to security concerns with its small 56 bit 
key size. The AES uses a key size of 128 bits or 256 bits. 


10.5 Public Key Systems 


A public key cryptosystem (Fig. 10.7) is an asymmetric key system where there is a 
separate key e, for encryption and d; decryption with e,. 4 d,. Martin Hellman and 
Whitfield Diffie invented it in 1976. The fact that a person is able to encrypt a 


Table. 10.3 DES encryption 


Step _ Description 


1 Expansion of the 32-bit half block to 48 bits (by duplicating half of the bits) 

a The 48-bit result is combined with a 48 bit subkey of the secret key using an XOR 
operation 

3 The 48-bit result is broken in to 8*6 bits and passed through 8 substitution boxes to 


yield 8*4 = 32 bits 
(This is the core part of the encryption algorithm) 


4 The 32-bit output is re-arranged according to a fixed permutation 
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Public Channel Hostile Attack 
(Insecure) (Enemy) 
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Message Encryption Decryption Message 
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Encryption Key Decryption Key 
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Fig. 10.7 Public key cryptosystem 


Table. 10.4 Public key encryption system 


Item Description 


1 It uses the concept of a key pair (e,, dx) 

a One half of the pair can encrypt messages and the other half can decrypt messages 

3 One key is private and one key is public 

4 The private key is kept secret and the public key is published (but associated with the 


trading partner) 


5 The key pair is associated with exactly one trading partner 


message does not mean that the person has sufficient information to decrypt mes- 
sages. The public key cryptosystem is based on the following (Table 10.4): 

The advantages and disadvantages of public key cryptosystems include 
(Table 10.5). 

The implementation of public-key cryptosystems is based on trapdoor one-way 
functions. A function f : X — Y is a trapdoor one way function if. 


— fis easy to compute 
— f' is difficult to compute 


— f'' is easy to compute if a trapdoor (secret information associated with the 
function) becomes available. 


A function satisfying just the first two conditions above is termed a one-way 
function. 


Examples of Trapdoor and One-way Functions 


(i) The function f : pg — n (where p and q are primes) is a one way function 
since it is easy to compute. However, the inverse function f' is difficult to 
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Table. 10.5 Advantages and disadvantages of public key cryptosystems 


Advantages Disadvantages 

Only the private key needs to be kept secret Public keys must be authenticated 
The distribution of keys for encryption is convenient, It is slow and uses more computer 
as everyone publishes their public key and the resources 


private key is kept private 

It provides message authentication as it allows the Security Compromise is possible (if 
use of digital signatures (which enables the recipient _ private key is compromised) 

to verify that the message is really from the particular 

sender) 


The sender encodes with the private key that is Loss of private key may be irreparable 
known only to sender. The receiver decodes with the —_ (unable to decrypt messages) 

public key and therefore knows that the message is 

from the sender 


Detection of tampering (digital signatures enable the 
receiver to detect whether message was altered in 
transit) 


Provides for non-repudiation 


compute problem for large n since there is no efficient algorithm to factorize 
a large integer into its prime factors (integer factorization problem). 

(ii) The function f, y : x — g* (mod N) is a one way function since it is easy to 
compute. However, the inverse function f * is difficult to compute as there is 
no efficient method to determine x from the knowledge of g* (mod N) and 
g and N (the discrete logarithm problem). 

(iii) The function fy y : x + x*(mod N) (where N = pq and p and q are primes) 
and kk” = 1 (mod N) is a trapdoor function. It is easy to compute but the 
inverse of f (the k root modulo N) is difficult to compute. However, if the 
trapdoor k’ is given then f can easily be inverted as (x*)* = x (mod N) 


10.5.1 RSA Public Key Cryptosystem 


Rivest, Shamir and Adleman proposed a practical public key cryptosystem 
(RSA) based on primality testing and integer factorization in the late 1970s. 
The RSA algorithm was filed as a patent (Patent No. 4,405, 829) at the U.S. Patent 
Office in December 1977. The RSA public key cryptosystem is based on the 
following assumptions: 


— It is straightforward to find two large prime numbers. 
— The integer factorization problem is infeasible for large numbers. 
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The algorithm is based on mod-n arithmetic where n is a product of two large 
prime numbers. 
The encryption of a plaintext message M to produce the cipher text C is given 


by: 
C = M*(modn) 


where e is the public encryption key, M is the plaintext, C is the cipher text, and n is 
the product of two large primes p and q. Both e and n are made public, and e is 
chosen such that 1<e<(n), where (7) is the number of positive integers that 
are relatively prime to n. 

The cipher text C is decrypted by 


M = C%(mod n) 


where d is the private decryption key that is known only to the receiver, and ed = 1 
(mod (n)) and d and ¢(n) are kept private. 

The calculation of (7) is easy if both p and q are known, as it is given by 
b(n) = (p — 1)(q — 1). However, its calculation for large n is infeasible if p and 
q are unknown. 


ed = 1(mod (n)) 
=> ed =1+k(n) forsome k € Z 


We discussed Euler’Theorem in Chap. 3, and this result states that if a and n are 
positive integers with gcd(a,n) = 1 then a®”) = 1(modn). Therefore, Mo) = 
1(modn) and M**) = 1(modn). The decryption of the cipher text is given by: 


C4(modn) = M“ (mod n) 
= M!+*$() (modn) 
= M'M'™ (mod n) 
= M.1(modn) 
= M(modn) 


10.5.2 Digital Signatures 


The RSA public-key cryptography may also be employed to obtain digital signa- 
tures. Suppose A wishes to send a secure message to B as well as a digital signature. 
This involves signature generation using the private key, and signature verification 
using the public key. The steps involved are: (Table 10.6): 
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Table. 10.6 Steps for A to Step Description 

send secure message and fl ny eee h 

signature to B uses B’s public key to encrypt the message 
2 A uses its private key to encrypt its signature 
3 A sends the message and signature to B 
4 B uses A’s public key to decrypt A’s signature 
3) B uses its private key to decrypt A’s message 


The National Institute of Standards and Technology (NIST) proposed an algo- 
rithm for digital signatures in 1991. The algorithm is known as the Digital Signature 
Algorithm (DSA) and later became the Digital Signature Standard (DSS). 


10.6 Review Questions 


1. Discuss the early ciphers developed by Julius Caesar and Augustus. How 
effective were they at that period in history, and what are their weaknesses 
today? 

2. Describe how the team at Bletchley Park cracked the German Enigma 
codes. 

3. Explain the differences between a public key cryptosystem and a private 
key cryptosystem. 

4. What are the advantages and disadvantages of private (symmetric) key 
cryptosystems? 

5. Describe the various types of symmetric key systems. 

6. What are the advantages and disadvantages of public key cryptosystems? 

7. Describe public key cryptosystems including the RSA public key 
cryptosystem. 

8. Describe how digital signatures may be generated. 


10.7 Summary 


This chapter provided a brief introduction to cryptography, which is the study of 
mathematical techniques that provide secrecy in the transmission of messages 
between computers. It was originally employed to protect communication between 
individuals, and today it is employed to solve security problems such as privacy and 
authentication over a communications channel. 
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It involves enciphering and deciphering messages, and theoretical results from 
number theory are employed to convert the original messages (or plaintext) into 
cipher text that is then transmitted over a secure channel to the intended recipient. 
The cipher text is meaningless to anyone other than the intended recipient, and the 
received cipher text is then decrypted to allow the recipient to read the message. 

A public key cryptosystem is an asymmetric cryptosystem. It has two different 
encryption and decryption keys, and the fact that a person has knowledge on how to 
encrypt messages does not mean that the person has sufficient information to 
decrypt messages. 

In a secret key cryptosystem the same key is used for both encryption and 
decryption. Anyone who has knowledge on how to encrypt messages has sufficient 
knowledge to decrypt messages, and it is essential that the key is kept secret 
between the two parties. 
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11.1. Introduction 


Coding theory is a practical branch of mathematics concerned with the reliable 
transmission of information over communication channels. It allows errors to be 
detected and corrected, which is essential when messages are transmitted through a 
noisy communication channel. The channel could be a telephone line, radio link or 
satellite link, and coding theory is applicable to mobile communications, and 
satellite communications. It is also applicable to storing information on storage 
systems such as the compact disc. 

It includes theory and practical algorithms for error detection and correction, and 
it plays an important role in modern communication systems that require reliable 
and efficient transmission of information. 

An error correcting code encodes the data by adding a certain amount of 
redundancy to the message. This enables the original message to be recovered if a 
small number of errors have occurred. The extra symbols added are also subject to 
errors, as accurate transmission cannot be guaranteed in a noisy channel. 

The basic structure of a digital communication system is shown in Fig. 11.1. It 
includes transmission tasks such as source encoding, channel encoding and mod- 
ulation; and receiving tasks such as demodulation, channel decoding and source 
decoding. 

The modulator generates the signal that is used to transmit the sequence of 
symbols b across the channel. The transmitted signal may be altered due to the fact 
that there is noise in the channel, and the signal received is demodulated to yield the 
sequence of received symbols r. 
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Fig. 11.1 Basic digital communication 


The received symbol sequence r may differ from the transmitted symbol 
sequence b due to the noise in the channel, and therefore a channel code is 
employed to enable errors to be detected and corrected. The channel encoder 
introduces redundancy into the information sequence u, and the channel decoder 
uses the redundancy for error detection and correction. This enables the transmitted 
symbol sequence # to be estimated. 

Shannon [1] showed that it is theoretically possible to produce an information 
transmission system with an error probability as small as required provided that the 
information rate is smaller than the channel capacity. 

Coding theory uses several results from pure mathematics, and so first we briefly 
discuss the mathematical foundations of coding theory. 


11.2. Mathematical Foundations 


Coding theory is built from the results of modern algebra, and it uses abstract 
algebraic structures such as groups, rings, fields and vector spaces. These abstract 
structures provide a solid foundation for the discipline, and the main structures used 
include vector spaces and fields. A group is a non-empty set with a single binary 
operation, whereas rings and fields are algebraic structures with two binary oper- 
ations satisfying various laws. A vector space consists of vectors over a field. 
We discussed these abstract mathematical structures in Chap. 6, and presented 
examples of each structure. The representation of codewords is by n-dimensional 
vectors over the finite field F,. A codeword vector v is represented as the n-tuple: 


v = (ao, 41,---»n—-1) 


where each a; € F,. The set of all n-dimensional vectors is the n-dimensional 
vector space F”, with q” elements. The addition of two vectors v and w, where 
Vv = (do, G1, ...- Gp-1) and w = (bo, by, .... by—1) iS given by 


vtw = (ao +bo,a, +b1,..--Gn—1 + Dn_1)- 
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The scalar multiplication of a vector v = (ao, dj,.... G,-1) € F", by a scalar 
B € F, is given by 


By = (Bao, Bar,.... Ban—1)- 


The set F”, is called the vector space over the finite field F, if the vector space 
properties above hold. A finite set of vectors v1, vo, ... vg is said to be linearly 
independent if 


Bivi + Byvo +--+ + Byve = 0 B, = B, =--- B, =0. 


Otherwise, the set of vectors v1, v2, ... Vg 1S said to be linearly dependent. 

The dimension (dim W) of a subspace WCY is k if there are k linearly inde- 
pendent vectors in W but every k + 1 vectors are linearly dependent. A subset of a 
vector space is a basis for V if it consists of linearly independent vectors, and its 
linear span is V (i.e. the basis generates V). We shall employ the basis of the vector 
space of codewords to create the generator matrix to simplify the encoding of the 
information words. The linear span of a set of vectors v,,V2,...,vs defined as 
Bivi + Bove + +++ + Byve. 


11.3 Simple Channel Code 


This section presents a simple example to illustrate the concept of an error cor- 
recting code. The example code presented is able to correct a single transmitted 
error only. 

We consider the transmission of binary information over a noisy channel that 
leads to differences between the transmitted sequence and the received sequence. 
The differences between the transmitted and received sequence are illustrated by 
underlining the relevant digits in the example. 


Sent 00101110 
Received 00000110 


Initially, it is assumed that the transmission is done without channel codes as 
follows: 


00101110 "3" 00000110 


Next, the use of an encoder is considered and a triple repetition-encoding scheme 
is employed. That is, the binary symbol 0 is represented by the code word 000, and 
the binary symbol | is represented by the code word 111. 


00101110 —|Encoder — 000000111000111111111000 
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Another words, if the symbol 0 is to be transmitted then the encoder emits the 
codeword 000, and similarly the encoder emits 111 if the symbol 1 is to be 
transmitted. Assuming that on average one symbol in four is incorrectly transmit- 
ted, then transmission with binary triple repetition may result in a received sequence 
such as 


000000111000111111111000 —| Channel — 010000011010111010111010 


The decoder tries to estimate the original sequence by using a majority decision 
on each 3-bit word. Any 3-bit word that contains more zeros than ones is decoded 
to 0, and similarly if it contains more ones than zero it is decoded to 1. The 
decoding algorithm yields 


01000001 1010111010111010 —) Decoder — 00101010 


In this example, the binary triple repetition code is able to correct a single error 
within a code word (as the majority decision is two to one). This helps to reduce the 
number of errors transmitted compared to unprotected transmission. In the first case 
where an encoder is not employed there are two errors, whereas there is just one 
error when the encoder is used. 

However, there are disadvantages with this approach in that the transmission 
bandwidth has been significantly reduced. It now takes three times as long to 
transmit an information symbol with the triple replication code than with standard 
transmission. Therefore, it is desirable to find more efficient coding schemes. 


11.4 Block Codes 


There were two code words employed in the simple example above: namely 000 
and 111. This is an example of a (n, k) code where the code words are of length 
n = 3, and the information words are of length k = 1 (as we were just encoding a 
single symbol 0 or 1). This is an example of a (3, 1) block code, and the objective of 
this section is to generalize the simple coding scheme to more efficient and pow- 
erful channel codes. 

The fundamentals of the g-nary (n, k) block codes (where q is the number of 
elements in the finite field F,) involve converting an information block of length 
k to a codeword of length n. Consider an information sequence up, U1, Uo, ......+-- of 
discrete information symbols where u; € {0, 1, ... g— 1} = Fy. The normal class of 
channel codes is when we are dealing with binary codes: i.e. g = 2. The information 
sequence is then grouped into blocks of length k as follows: 


Uguyu2.. Ug—1 URUK + {UK+2-~-U2k-1 — U2kU2k4 1U2kK4+2-. -U3k-1 °° °° 
—— esa Wi i Oa eee — 
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Each block is of length k (i.e. the information words are of length k), and it is 
then encoded separately into codewords of length n. For example, the block uu, 
+1UK42 «-. U2k-1- 

is encoded to the code word byby+jDn+2 ... Doy—1. 

of length n where b; € F,. Similarly, the information word ugujuz ... Uz is 
uniquely mapped to a code word bobybz ...By-1 of tength n as follows. 


(Up U2. . .Up—1) —|Encoder — (bobi bo. . .bn-1). 


These code words are then transmitted across the communication channel and 
the received words are then decoded. The received word r = (rorjro ... Tp-1) 18 
decoded into the information word U = (doit). . .,_1). 


(Torino. ‘ Tn—1) —|Decoder > (Hip it it. 7 itp). 


Strictly speaking the decoding is done in two steps with the received n-block 
word r first decoded to the n-block codeword b’. This is then decoded into the k- 
block information word 0. The encoding, transmission and decoding of an (n, k) 
block may be summarized as follows (Fig. 11.2): 

A lookup table may be employed for the encoding to determine the code word 
b for each information word u. However, the size of the table grows exponentially 
with increasing information word length k, and so this is inefficient due to the large 
memory size required. We shall discuss later how a generator matrix provides an 
efficient encoding and decoding mechanism. 


Notes 


(i) The codeword is of length n. 

(ii) The information word is of length k. 

(iii) The codeword length n is larger than the information word length k. 

(iv) A block (n, k) code is a code in which all codewords are of length n and all 
information words are of length k. 

(v) The number of possible information words is given by M = a (where each 
information symbol can take one of q possible values and the length of the 
information word is k). 


. Channel : Ghamnct é Channel 
> : >| Modulator [7~7~~~-~ > |Demodulator| i . 
Encoder Decoder 


(UplUy ...Uz) (bobby ..-b, 4) (Folin Tn) (tot, ..-0y.4) 


Fig. 11.2 Encoding and decoding of an (n, k) block 
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(vi) The code rate R in which information is transmitted across the channel is 
given by 


k 
R=-. 
n 


(vii) The weight of a codeword is b = (bob, b2 ... b,_1) is given by the number of 
non-zero components of b. That is, 


wt(b) = |{i:b; 40, 0<i<n}I. 


(viii) The distance between two codewords b = (bobyb2 ... by.,) and b' = (bo'by' 
b,' ...b,—1') measures how close the codewords b and b’ are to each other. It 
is given by the Hamming distance: 


dist(b, b’) = |{i: bj) ABI, O<i<n}I. 


(ix) The minimum Hamming distance for a code B consisting of M codewords 
by, ....., by is given by 


d = min{dist(b, b') : where b 4 b’}. 


(x) The (n, k) block code B = {by, ...., by} with M (= q') codewords of length 
n and minimum Hamming distance d is denoted by B(n, k, d). 


11.4.1 Error Detection and Correction 


The minimum Hamming distance offers a way to assess the error detection and 
correction capability of a channel code. Consider two codewords b and b' of an (n, 
k) block code B(n, k, d). 

Then, the distance between these two codewords is greater than or equal to the 
minimum Hamming distance d, and so errors can be detected as long as the erro- 
neously received word is not equal to a codeword different from the transmitted 
code word. 

That is, the error detection capability is guaranteed as long as the number of 
errors is less than the minimum Hamming distance d, and so the number of 
detectable errors is d — 1. 
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Fig. 11.3 Error correcting capability sphere 


Any two codewords are of distance at least d and so if the number of errors is 
less than “/> then the received word can be properly decoded to the codeword b. 
That is, the error correction capability is given by: 


cor = — : 
2 

An error correcting sphere (Fig. 11.3) may be employed to illustrate the error 
correction of a received word to the correct codeword b. This may be done when all 
received words are within the error correcting sphere with radius p (< 8 

If the received word r is different from b in less than “/5 positions, then it is 
decoded to b as it is more than“/, positions from the next closest codeword. That is, 
b is the closest codeword to the received word r (provided that the error-correcting 
radius is less than ay, 


11.5 Linear Block Codes 


Linear block codes have nice algebraic properties, and the codewords in a linear 
block code are considered to be vectors in the finite vector space F”,. The repre- 
sentation of codewords by vectors allows the nice algebraic properties of vector 
spaces to be used, and this simplifies the encoding of information words as a 
generator matrix may be employed to create the codewords. 

An (n, k) block code B(n, k, d) with minimum Hamming distance d over the 
finite field F, is called linear if B(n, k, d) is a subspace of the vector space F", of 
dimension k. The number of codewords is then given by 


M= a. 
The rate of information (R) through the channel is given by 


k 
R=-. 
n 
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Clearly, since B(n, k, d) is a subspace of F”, any linear combination of the 
codewords (vectors) will be a codeword. That is, for the codewords bj,bo,...,b, we 
have that 


ab; +o2b2 +...+ a,b, € B(n,k, d) 


where 1, 012,...,%, € Fy and bj,bo,...,b, € Bin, k, d). 

Clearly, the n-dimensional zero row vector (0, 0, ..., 0) is always a codeword, 
and so (0, 0, ..., 0) € Bi, k, d). The minimum Hamming distance of a linear block 
code B(n, k, d) is equal to the minimum weight of the non-zero codewords: That is, 


d= min {dist(b, b’)} = min wt(b). 
aa Las )} sain vith) 


In summary, an (n, k) linear block code B(n, k, d) is 


1. A subspace of F”,, 

2. The number of codewords is M = a. 

3. The minimum Hamming distance d is the minimum weigh of the non-zero 
codewords. 


The encoding of a specific k-dimensional information word u = (Uo, Wy, ... U1) 
to a n-dimensional codeword b = (bo, by, ..., b;-1) may be done efficiently with a 
generator matrix. First, a basis {go, 91, ... 8x-1} of the k-dimensional subspace 
spanned by the linear block code is chosen, and this consists of k linearly inde- 
pendent n-dimensional vectors. Each basis element g; (whereO < i < k-l)isa 
n-dimensional vector: 


gi = (8i,0 > Sisk ++ ++ 8inn-1 ). 


The corresponding codeword b = (bo, bj,...., b,-1) is then a linear combination 
of the information word with the basis elements. That is, 


b = woo + mig) + + FUK-1 84-1 


where each information symbol vu; € F,. The generator matrix G is then constructed 
from the k linearly independent basis vectors as follows (Fig. 11.4): 

The encoding of the k-dimensional information word u to the n-dimensional 
codeword b involves matrix multiplication (Fig. 11.5): 

This may also be written as 


b =uG. 
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Fig. 11.4 Generator matrix 
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Fig. 11.5 Generation of codewords 


Clearly, all M = q‘ codewords b € B(n, k, d) can be generated according to this 
rule, and so the matrix G is called the generator matrix. The generator matrix 
defines the linear block code Bin, k, d). 

There is an equivalent k x n generator matrix for B(n, k, d) defined as 


G=Ik| Agn—z 


where I; is the k x k identity matrix (Fig. 11.6): 
The encoding of the information word u yields the codeword b such that the first 
k symbols b; of b are the same as the information symbols u;0 <i < k. 


b=uG= (u |u Agn-k): 


The remaining m=n — k symbols are generated from uA;,, and the last 
m symbols are the m parity check symbols. These are attached to the information 
vector u for the purpose of error detection and correction. 
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Fig. 11.6 Identity matrix (— i 


(kx k 1 O O see, (0 
O 1 O ... O 
O (0) 1 ... O 
L= 
Oe OD aks i, 


11.5.1 Parity Check Matrix 


The linear block code Bn, k, d) with generator matrix G = (I, | Agn-~) may be 
defined equivalently by the (7 — k) x n parity check matrix H, where this matrix is 
defined as 


HS (-AL.-«l Tk): 


The generator matrix G and the parity check matrix H are orthogonal: i.e. 
HG? = 0,-4.k- 


The parity check orthogonality property holds if and only if the vector belongs to 
the linear block code. That is, for each code vector in b € B(n, k, d) we have 


Hb! = On—k, Al 


and vice verse whenever the property holds for a vector r, then ris a valid codeword 
in Bim, k, d). We present an example of a parity check matrix in Example 11.1 
below. 


11.5.2 Binary Hamming Code 


The Hamming code is a linear code that has been employed in dynamic random 
access memory to detect and correct deteriorated data in memory. The generator 
matrix for the B(7, 4, 3) binary Hamming code is given by (Fig. 11.7): 

The information words are of length k = 4 and the codewords are of length 
n = 7. For example, it can be verified by matrix multiplication that the information 
word (0, 0, 1, 1) is encoded into the codeword (0, 0, 1, 1, 0, 0, 1). 

That is, three parity bits 001 have been added to the information word (0, 0, 1, 1) 
to yield the codeword (0, 0, 1, 1, 0, 0, 1). 
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Fig. 11.7 Hamming code B(7, 4, 3) generator matrix 


The minimum Hamming distance is d = 3, and the Hamming code can detect up 
to two errors, and it can correct one error. 


Example 11.1 (Parity Check Matrix—Hamming Code) 

The objective of this example is to construct the Parity Check Matrix of the Binary 
Hamming Code (7, 4, 3), and to show an example of the parity check orthogonality 
property. 

First, we construct the parity check matrix H which is given by H = (—A;,._;|Jn—x) 
or another words H = (—Aj,|I3). We first note that 


2 O14 
Aw=!11 1 9 Ag=| 10 1-4 
al 1101 


Therefore, H is given by 


Oo -1 -1 -1 1 0 0 
H=j;-1 0 -1 -1 0 1 0 
-1 -!l1 0 -1 00 1 


We noted that the encoding of the information word u = (0011) yields the code- 
word b = (0,011,001). Therefore, the calculation of Hb" yields (recalling that 
addition is modulo two): 


0 
0 
0 -1 -1 -1 10 0\J]1 0 
Hb' =] -1 O -1 -1 01 0 1}/={0 
-1 -1 0 -10 0 1/40 0 

0 

1 
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11.5.3 Binary Parity-Check Code 


The binary parity-check code is a linear block code over the finite field Fy. The code 
takes a k-dimensional information word u = (uo, uy, ... Ug—1) and generates the 
codeword b = (bo, by, ...., by-1, by) where u; = b; (0 < i < k—- 1) and b; is the 
parity bit chosen so that the resulting codeword is of even parity. That is, 


k-1 
by = Up tuy +--+ Fu, = Uj. 


ll 
o 


11.6 Miscellaneous Codes in Use 


There are many examples of codes in use such as repetition codes (such as the triple 
replication code considered earlier in Sect. 11.3); parity check codes where a parity 
symbol is attached such as the binary parity-check code; Hamming codes such as 
the (7, 4) code that was discussed in Sect. 11.5.2, and which has been applied for 
error correction of faulty memory. 

The Reed—Muller codes form a class of error correcting codes that can correct 
more than one error. Cyclic codes are special linear block codes with efficient 
algebraic decoding algorithms. The BCH codes are an important class of cyclic 
codes, and the Reed Solomon codes are an example of a BCH code. 

Convolution codes have been applied in the telecommunications field, for 
example, in GSM, UMTS and in satellite communications. They belong to the class 
of linear codes, but also employ a memory so that the output depends on the current 
input symbols and previous input. For more detailed information on coding theory 
see [2]. 


11.7 Review Questions 


. Describe the basic structure of a digital communication system. 
. Describe the mathematical structure known as the field. Give examples of 
fields. 

3. Describe the mathematical structure known as the ring and give examples 
of rings. Give examples of zero divisors in rings. 

4. Describe the mathematical structure known as the vector space and give 
examples. 

5. Explain the terms linear independence and linear dependence and a basis. 

6. Describe the encoding and decoding of an (n, k) block code where an 

information word of length k is converted to a codeword of length n. 


Ne 
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7. Show how the minimum Hamming distance may be employed for error 
detection and error correction. 

8. Describe linear block codes and show how a generator matrix may be 
employed to generate the codewords from the information words. 


11.8 Summary 


Coding theory is the branch of mathematics that is concerned with the reliable 
transmission of information over communication channels. It allows errors to be 
detected and corrected, and this is extremely useful when messages are transmitted 
through a noisy communication channel. This branch of mathematics includes 
theory and practical algorithms for error detection and correction. 

The theoretical foundations of coding theory were considered, and its founda- 
tions lie in abstract algebra including group theory, ring theory, fields and vector 
spaces. The codewords are represented by n-dimensional vectors over a finite field 
Fo. 

An error correcting code encodes the data by adding a certain amount of 
redundancy to the message so that the original message can be recovered if a small 
number of errors have occurred. 

The fundamentals of block codes were discussed where an information word is 
of length k and a codeword is of length n. This led to the linear block codes B(n, k, 
d) and a discussion on error detection and error correction capabilities of the codes. 

The goal of this chapter was to give a flavour of coding theory, and the reader is 
referred to more specialised texts (e.g. [2]) for more detailed information. 
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Language Theory and Semantics 


12.1. Introduction 


There are two key parts to any programming language, and these are its syntax and 
semantics. The syntax is the grammar of the language and a program needs to be 
syntactically correct with respect to its grammar. The semantics of the language is 
deeper, and determines the meaning of what has been written by the programmer. 

The difference between syntax and semantics may be illustrated by an example 
in a natural language. A sentence may be syntactically correct but semantically 
meaningless, or a sentence may have semantic meaning but be syntactically 
incorrect. For example, consider the sentence: 


“T will go to Dublin yesterday”. 


Then this sentence is syntactically valid but semantically meaningless. Similarly, 
if a speaker utters the sentence “Me Dublin yesterday” we would deduce that the 
speaker had visited Dublin the previous day even though the sentence is syntacti- 
cally incorrect. 

The semantics of a programming language determines what a syntactically valid 
program will compute. A programming language is therefore given by 


Programming Language = Syntax + Semantics 


Many programming languages have been developed over the last sixty years 
including Plankalktil which was developed by Zuse in the 1940s; Fortran developed 
by IBM in the 1950s; Cobol was developed by a committee in the late 1950s; Algol 
60 and Algol 68 were developed by an international committee in the 1960s; Pascal 
was developed by Wirth in the early 1970s; Ada was developed for the US military 
in the late 1970s; the C language was developed by Richie and Thompson at Bell 
Labs in the 1970s; C++ was developed by Stroustrup at Bell Labs in the early 
1980s and Java developed by Gosling at Sun Microsystems in the mid-1990s. 
A short description of a selection of programming languages in use is in [1]. 
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A programming language needs to have a well-defined syntax and semantics, and 
the compiler preserves the semantics of the language. Compilers are programs that 
translate a program that is written in some programming language into another form. 
It involves syntax analysis and parsing to check the syntactic validity of the pro- 
gram; semantic analysis to determine what the program should do; optimization to 
improve the speed and performance; and code generation in some target language. 

Alphabets are a fundamental building block in language theory, as words and 
language are generated from alphabets. They are discussed in the next section. 


12.2 Alphabets and Words 


An alphabet is a finite non-empty set A, and the elements of A are called letters. For 
example, consider the usual alphabet which is a set A that consists of the letters 
from a to z. 


Words are finite strings of letters, and a set of words is generated from the alphabet. 
For example, the alphabet A = {a, b} generates the following set of words: 


{e,a, b, aa, ab, bb, ba, aaa, bbb,.......}. 


(e denotes the empty word.) 
Each non-empty word consists of an ordered list of one or more letters, and the 
set of words of length two consists of all ordered lists of two letters. It is given by 


A? = {aa, ab, bb, ba}. 
Similarly, the set of words of length three is given by 
A= {aaa, aab, abb, aba, baa, bab, bbb, bba}. 


The set of all words over the alphabet A is given by the positive closure A*, and 
it is defined by 


At =AUA*UA?U...... =a". 
Given any two words w, = a 1d3... a, and wz = b,b3... b, then the concatenation 
of w, and wy is given by 
W=W1wW2 = 41a2.. ab b>. , .b,. 


The empty word is a word of length zero and is denoted by e. Clearly, ew = 
we = w for all w and so ¢ is the identity element under the concatenation operation. 
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A® is used to denote the set containing the empty word {e}, and the closure 
A*(= A* U{e}) denotes the infinite set of all words over A (including empty 
words). It is defined as 


A= Lar. 


n=1 


The mathematical structure (A’, A, €) forms a monoid,’ where “ is the con- 
catenation operator for words and the identity element is ¢. The length of a word 
w is denoted by |w| and the length of the empty word is zero: i.e. |e] = 0. 

A subset L of A’ is termed a formal language over A. Given two languages L,, Lz 
then the concatenation (or product) of L; and L, is defined by 


L,L2 = {w|w = wiw2 where w; € L; and w2 € Lp}. 


The positive closure of L and the closure of L may also be defined as 


lord x 
Lt = L" L* = L". 


12.3. Grammars 


A formal grammar describes the syntax of a language, and we distinguish between 
concrete and abstract syntax. Concrete syntax describes the external appearance of 
programs as seen by the programmer, whereas abstract syntax aims to describe the 
essential structure of programs rather than the external form. In other words, 
abstract syntax aims to give the components of each language structure while 
leaving out the representation details (e.g. syntactic sugar). Backus Naur form 
(BNF) notation is often used to specify the concrete syntax of a language. 
A grammar consists of the following: 


— A finite set of terminal symbols; 

— A finite set of nonterminal symbols; 
— A set of production rules; 

— A start symbol. 


A formal grammar generates a formal language, which is set of finite length 
sequences of symbols created by applying the production rules of the grammar. The 
application of a production rule involves replacing symbols at the left-hand side of 


‘Recall from Chap. 6 that a monoid (M, *, e) is a structure that is closed and associative under the 
binary operation ‘*’, and it has an identity element ‘e’. 
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Fig. 12.1 Noah Chomsky. 
Creative commons 


the rule with the symbols on the right-hand side of the rule. The formal language 
then consists of all words consisting of terminal symbols that are reached by a 
derivation (i.e. the application of production rules) starting from the start symbol of 
the grammar. 

A construct that appears on the left-hand side of a production rule is termed a 
nonterminal, whereas a construct that only appears on the right-hand side of a 
production rule is termed a terminal. The set of nonterminals N is disjoint from the 
set of terminals A. 

The theory of the syntax of programming languages is well established, and 
programming languages have a well-defined grammar that allows syntactically 
valid programs to be derived from the grammars. 

Chomsky” (Fig. 12.1) is a famous linguisit who classified a number of different 
types of grammar that occur. The Chomsky hierarchy (Table 12.1) consists of four 
levels including regular grammars; context free grammars; context sensitive 
grammars and unrestricted grammars. The grammars are distinguished by the 
production rules, which determine the type of language that is generated. 

Regular grammars are used to generate the words that may appear in a pro- 
gramming language. This includes the identifiers (e.g. names for variables, func- 
tions and procedures); special symbols (e.g. addition, multiplication, etc.) and the 
reserved words of the language. 

A rewriting system for context free grammars is a finite relation between N and 
(AUN)*: i.e. a subset of N x (AUN)": A production rule <N> — w is one ele- 
ment of this relation, and is an ordered pair (<N> , w) where w is a word consisting 
of zero or more terminal and nonterminal letters. This production rule means 
that <N> may be replaced by w. 


?Chomsky made important contributions to linguistics and the theory of grammars. He is more 
widely known today as a critic of United States foreign policy. 
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Table 12.1 Chomsky hierarchy of grammars 


Grammar type 


Description 


Type 0 Type 0-grammars include all formal grammars. They have production rules 

grammar of the form « — B where o and f are strings of terminals and nonterminals. 
They generate all languages that can be recognized by a Turing machine 
(discussed in Chap. 7) 

Type 1 These grammars generate the context sensitive languages. They have 

grammar production rules of the form «AB — ayB where A is a nonterminal and «, B 

(context and y are strings of terminals and nonterminals. They generate all languages 

sensitive) that can be recognized by a linear bounded automaton* 

Type 2 These grammars generate the context free languages. These are defined by 

grammar rules of the form A — y where A is a nonterminal and y is a string of 


(context free) 


terminals and nonterminals. These languages are recognized by a pushdown 


automaton’ and are used to define the syntax of most programming 


languages 
Type 3 These grammars generate the regular languages (or regular expressions). 
grammar These are defined by rules of the form A — a or A — aB where A and B are 
(regular nonterminals and a is a single terminal. A finite state automaton recognizes 
grammars) these languages (discussed in Chap. 7), and regular expressions are used to 
define the lexical structure of programming languages 
12.3.1 Backus Naur Form 


Backus Naur form® (BNF) provides an elegant means of specifying the syntax of 
programming languages. It was originally employed to define the grammar for the 
Algol-60 programming language [2], and a variant was used by Wirth to specify the 
syntax of the Pascal programming language. BNF is widely used and accepted 
today as the way to specify the syntax of programming languages. 

BNF specifications essentially describe the external appearance of programs as 
seen by the programmer. The grammar of a context free grammar may then be input 
into a parser (e.g. Yacc), and the parser is used to determine if a program is 
syntactically correct or not. 

A BNF specification consists of a set of production rules with each production 
tule describing the form of a class of language elements such expressions, state- 
ments and so on. A production rule is of the form: 


<symbol > ::= <expression withsymbols > 


3A linear bounded automaton is a restricted form of a nondeterministic Turing machine in which a 
limited finite portion of the tape (a function of the length of the input) may be accessed. 

“A pushdown automaton is a finite automaton that can make use of a stack containing data, and it 
is discussed in Chap. 7. 

Backus Naur Form is named after John Backus and Peter Naur. It was created as part of the 
design of the Algol 60 programming language, and is used to define the syntax rules of the 
language. 
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where <symbol> is a nonterminal, and the expression consists of sequenc of ter- 
minal and nonterminal symbols. A construct that has alternate forms appears more 
than once, and this is expressed by sequences separated by the vertical bar “|” 
(which indicates a choice). In other word, there is more than one possible substi- 
tution for the symbol on the left-hand side of the rule. Symbols that never appear on 
the left-hand side of a production rule are called terminals. 

The following example defines the syntax of various statements in a sample 
programming language: 

<loop statement> :: = <while loop> | <for loop> 

<while loop> :: = while ( <condition> ) <statement> 

<for loop> :: =f or (<expression>) <statement> 

<statement > :: = <assignment statement> | <loop statement > 

< assignment statement> :: = < variable>: = < expression> 

This is a partial definition of the syntax of various statements in the language. It 
includes various nonterminals such as (<loop statement> , <while loop> and so 
on. The terminals include ‘while’, ‘for’, ‘: = ’, ‘(‘ and “)’. The production rules 
for <condition> and <expression > are not included. 

The grammar of a context free language (e.g. LL(1), LL(A), LRC), LR(A)) 
grammar expressed in BNF notation) may be translated by a parser into a parse 


table. The parse table may then be employed to determine whether a particular 
program is valid with respect to its grammar. 


Example 12.1 (Context Free Grammar) 
The example considered is that of parenthesis matching in which there are two 
terminal symbols and one nonterminal symbol 


S—SS 
S = (S) 
S— (). 


Then by starting with S and applying the rules we can construct 


S— SS — (S)S > (()S > (O)0- 


Example 12.2 (Context Free Grammar) 

The example considered is that of expressions in a programming language. The 
definition is ambiguous as there is more than one derivation tree for some expres- 
sions (e.g. there are two parse trees for the expression 5 x 3 + | discussed below). 
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<expr> ::= <numeral> | (<expr>) 
| (<expr> <operator> <expr>) 
<operator> := + |— | x |/ 
<digit> :=O|1]|....|9 


<numeral>::=<digit> |<digit><numeral> 


Example 12.3 (Regular Grammar) 
The definition of an identifier in most programming languages is similar to 


<identifier> ::= <let> <letdig> 

<letdig> ::= <let> | <dig> | « 

<letdig> ::= <let> <letdig> | <dig> <letdig> 
<let>::=a|b|c|....|z 


<dig> =O |1]|....|9 


12.3.2 Parse Trees and Derivations 


Let A and N be the terminal and nonterminal alphabet of a rewriting system and 
let<X> —w be a production. Let x be a word in (AUN)* with 
x =u <X>v for some words u,v € (AUN)’. Then x is said to directly yield uwv 
and this is written as x > uwv. 

This single substitution (=) can be extended by a finite number of productions 
(= *), and this gives the set of words that can be obtained from a given word. This 
derivation is achieved by applying several production rules (one production rule is 
applied at a time) in the grammar. 

That is, given x, y € (AUN)* then x yields y (or y is a derivation of x) if x = y, or 
there exists a sequence of words w,,W2,....W, € (A UN)* such that x = wi, y = Wy, 
and w; > wj+1 for! < i < n-—1. This is written as x >* y. 

The expression grammar presented in Example 12.2 is ambiguous, and this 
means that an expression such as 5 x 3 + 1 has more than one interpretation. 
(Figs. 12.2, 12.3). It is not clear from the grammar whether multiplication is per- 
formed first and then addition, or whether addition is performed first and then 
multiplication. 

The first parse tree is given in Fig. 12.2, and the interpretation of the first parse 
tree is that multiplication is performed first and then addition (this is the normal 
interpretation of such expressions in programming languages as multiplication is a 
higher precedence operator than addition). 

The interpretation of the second parse tree is that addition is performed first and 
then multiplication (Fig. 12.3). It may seem a little strange that one expression has 
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<expr> 


<expr> <operator> <expr> 
<expr> <operator> <expr> <numeral> 
<numeral> <numeral> <digit> 
<digit> <digit> 
5 x 3 + 1 
Fig. 12.2 Parse tree 5 x 3 + 1 (mltiplication first) 
Fig. 12.3. Parse Tree <expr> 
5 x 34+ I (addition first) ee a 
<expr> <operator> <expr> 
<numeral> <expr> <operator> <expr> 
| | 
<digit> eee <numeral> 
<digit> <digit> 
| | 
5 x 3 + 1 


two parse trees, and it shows that the grammar is ambiguous. This means that there 
is a choice for the compiler in evaluating the expression, and the compiler needs to 
assign the right meaning to the expression. One solution would be for the language 
designer to alter the definition of the grammar to remove the ambiguity. 


12.4 Programming Language Semantics 


The formal semantics of a programming language is concerned with defining the 
actual meaning of a language. Language semantics is deeper than syntax, and the 
theory of the syntax of programming languages is well established. A programmer 
writes a program according to the rules of the language. The compiler first checks 
the program for synyactic correctness: i.e. it determines whether the program as 
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Table 12.2 Programming language semantics 


Approach Description 
Axiomatic This involves giving meaning to phrases of the language using logical 
semantics axioms 


It employs pre and post condition assertions to specify what happens 
when the statement executes. The relationship between the initial 
assertion and the final assertion essentially gives the semantics of the code 


Operational This approach describes how a valid program is interpreted as sequences 
semantics of computational steps. These sequences then define the meaning of the 
program 


An abstract machine (SECD machine) may be defined to give meaning to 
phrases, and this is done by describing the transitions they induce on 
states of the machine 


Denotational This approach provides meaning to programs in terms of mathematical 
semantics objects such as integers, tuples and functions 
Each phrase in the language is translated into a mathematical object that is 
the denotation of the phrase 


written is valid according to the rules of the grammar of the language. If the 
program is syntactically correct, then the compiler determines the meaning of what 
has been written and generates the corresponding machine code.° 

The compiler must preserve the semantics of the language: i.e. the semantics are 
not defined by the compiler, but rather the function of the compiler is to preserve 
the semantics of the language. Therefore, there is a need to have an unambiguous 
definition of the meaning of the language independently of the compiler, and the 
meaning is then preserved by the compiler. 

A program’s syntax’ gives no information as to the meaning of the program, and 
therefore there is a need to supplement the syntactic description of the language 
with a formal unambiguous definition of its semantics. 

It is possible to utter syntactically correct but semantically meaningless sen- 
tences in a natural language. Similarly, it is possible to write syntactically correct 
programs that behave in quite a different way from the intention of the programmer. 

The formal semantics of a language is given by a mathematical model that 
describes the possible computations described by the language. There are three 
main approaches to programming language semantics namely axiomatic semantics, 
operational semantics and denotational semantics (Table 12.2). 

There are several applications of programming language semantics including 
language design, program verification, compiler writing and language standard- 
ization. The three main approaches to semantics are described in more detail below. 


Of course, what the programmer has written may not be what the programmer had intended. 
’There are attribute (or affix) grammars that extend the syntactic description of the language with 
supplementary elements covering the semantics. The process of adding semantics to the syntactic 
description is termed decoration. 
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12.4.1 Axiomatic Semantics 


Axiomatic semantics gives meaning to phrases of the language by describing the 
logical axioms that apply to them. It was developed by C.A.R. Hoare® in a famous 
paper “An axiomatic basis for computer programming” [3]. His axiomatic theory 
consists of syntactic elements, axioms and rules of inference. 

The well-formed formulae that are of interest in axiomatic semantics are pre-post 
assertion formulae of the form P{a}Q, where a is an instruction in the language and 
P and Q are assertions: i.e. properties of the program objects that may be true or false. 

An assertion is essentially a predicate that may be true in some states and false in 
other states. For example, the assertion (x — y > 5) is true in the state in which the 
values of x and y are 7 and | respectively, and false in the state where x and y have 
values 4 and 2. 

The pre and post condition assertions are employed to specify what happens 
when the statement executes. The relationship between the initial assertion and the 
final assertion gives the semantics of the code statement. The pre and post condition 
assertions are of the form: 


P{ahQ. 


The precondition P is a predicate (input assertion), and the postcondition Q is a 
predicate (output assertion). The braces separate the assertions from the program 
fragment. The well-formed formula P{a}Q is itself a predicate that is either true or 
false. 

This notation expresses the partial correctness” of a with respect to P and Q, and 
its meaning is that if statement a is executed in a state in which the predicate P is 
true and execution terminates, then it will result in a state in which assertion Q is 
satisfied. 

The axiomatic semantics approach is described in more detail in [5], and the 
axiomatic semantics of a selection of statements is presented below. 


e Skip 


The skip statement does nothing and whatever condition is true on entry to the 
command is true on exit from the command. It’s meaning is given by 


P{skip}P. 


SHoare was influenced by earlier work by Floyd on assigning meanings to programs using 
flowcharts [4]. 

Total correctness is expressed using {P}a{Q} amd program fragment a is totally correct for 
precondition P and postcondition Q if and only if whenever a is executed in any state in which P is 
satisfied then execution terminates, and the resulting state satisfies Q. 
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e Assignment 


The meaning of the assignment statement is given by the axiom: 
Pi {x := e}P. 


The meaning of the assignment statement is that P will be true after execution of 
the assignment statement if and only if the predicate P*, with the value of x replaced 
by e in P is true before execution (since x will contain the value of e after 
execution). 

The notation P*, denotes the expression obtained by substituting e for all free 
occurrences of x in P. 


e Compound 


The meaning of the conditional command is 


P{Si}0.0{S2}R 
P{S1; S2}R : 


The compound statement involves the execution of S; followed by the execution 
of Sz. The meaning of the compound statement is that R will be true after the 
execution of the compound statement $1; Sj provided that P is true, if it is estab- 
lished that Q will be true after the execution of S; provided that P is true, and that 
R is true after the execution of Sz provided Q is true. 

There needs to be at least one rule associated with every construct in the lan- 
guage in order to give its axiomatic semantics. The semantics of other programming 
language statements such as the ‘while’ statement and the ‘if’ statement are 
described in [5]. 


12.4.2 Operational Semantics 


The operational semantics definition is similar to an interpreter, where the semantics 
of a language are expressed by a mechanism that makes it possible to determine the 
effect of any program in the language. The meaning of a program is given by the 
evaluation history that an interpeter produces when it interprets the program. The 
interpreter may be close to an executable programming language or it may be a 
mathematical language. 

The operational semantics for a programming language describes how a valid 
program is interpreted as sequences of computational steps. The evaluation history 
then defines the meaning of the program, and this is a sequence of internal inter- 
preter configurations. 
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One early use of operational semantics was the work done by John McCarthy in 
the late 1950s on the semantics of LISP in terms of the lambda calculus. The use of 
lambda calculus allows the meaning of a program to be expressed using a math- 
ematical interpreter, and this offers precision through the use of mathematics. 

The meaning of a program may be given in terms of a hypothetical or virtual 
machine that performs the set of actions that corresponds to the program. An 
abstract machine (SECD machine!) may be defined to give meaning to phrases in 
the language, and this is done by describing the transitions that they induce on 
states of the machine. 

Operational semantics give an intuitive description of the programming language 
being studied, and its descriptions are close to real programs. It can play a useful 
role as a testing tool during the design of new languages, as it is relatively easy to 
design an interpreter to execute the description of example programs. This allows 
the effects of new languages or new language features to be simulated and studied 
through actual execution of the semantic descriptions prior to writing a compiler for 
the language. Another words, operational semantics can play a role in rapid pro- 
totyping during language design, and to get early feedback on the suitability of the 
language. 

One disadvantage of the operational approach is that the meaning of the lan- 
guage is understood in terms of execution: i.e. in terms of interpreter configurations, 
rather than in an explicit machine independent specification. An operational 
description is just one way to execute programs. Another disadvantage is that the 
interpreters for non-trivial languages often tend to be large and complex. A more 
detailed account of operational semantics is in [6, 7]. 


12.4.3 Denotational Semantics 


Denotational semantics [7] expresses the semantics of a programming language by 
a translation schema that associates a meaning (denotation) with each program in 
the language. It maps a program directly to its meaning, and it was originally called 
mathematical semantics as it provides meaning to programs in terms of mathe- 
matical values such as integers, tuples and functions. That is, the meaning of a 
program is a mathematical object, and an interpreter is not employed. Instead, a 
valuation function is employed to map a program directly to its meaning, and the 
denotational description of a programming language is given by a set of meaning 
functions M associated with the constructs of the language (Fig. 12.4). 

Each meaning function is of the form My : T — Dr where T is some construct 
in the language and Dy is some semantic domain. Many of the meaning functions 
will be ‘higher order’: i.e. functions that yield functions as results. The signature of 
the meaning function is from syntactic domains (i.e. T) to semantic domains (i.e. 


‘This virtual stack based machine was originally designed by Peter Landin to evaluate lambda 
calculus expressions, and it has since been used as a target for several compilers. 


12.4 Programming Language Semantics 203 


Meaning Function 


Mathematical 
Denotation 
Fig. 12.4 Denotational semantics 


Dy). A valuation map V7 : T — B may be employed to check the static semantics 
prior to giving a meaning of the language construct.'! 

A denotational definition is more abstract than an operational definition. It does 
not specify the computational steps and its exclusive focus is on the programs to the 
exclusion of the state and other data elements. The state is less visible in denota- 
tional specifications. 

It was developed by Christopher Strachey and Dana Scott at the Programming 
Research Group at Oxford, England in the mid-1960s, and their approach to 
semantics is known as the Scott-Strachey approach [8]. It provided a mathematical 
foundation for the semantics of programming languages. 

Dana Scott’s contributions included the formulation of domain theory, and this 
allowed programs containing recursive functions and loops to be given a precise 
semantics. Each phrase in the language is translated into a mathematical object that 
is the denotation of the phrase. Denotational Semantics has been applied to lan- 
guage design and implementation. 


12.5 Lambda Calculus 


Functions (discussed in Chap. 2) are an essential part of mathematics, and they play 
a key role in specifying the semantics of programming language constructs. We 
discussed partial and total functions in Chap. 2, and a function was defined as a 
special type of relation, and simple finite functions may be defined as an explicit set 
of pairs: e.g. 


fA{(a, 1), (6,2); (c,3)}- 


However, for more complex functions there is a need to define the function more 
abstractly, rather than listing all of its member pairs. This may be done in a similar 
manner to set comprehension, where a set is defined in terms of a characteristic 
property of its members. 


"This is similar to what a compiler does in that if errors are found during the compilation phase, 
the compiler halts and displays the errors and does not continue with code generation. 
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Functions may be defined (by comprehension) through a powerful abstract 
notation known as lambda calculus. This notation was introduced by Alonzo 
Church in the 1930s to study computability, and lambda calculus provides an 
abstract framework for describing mathematical functions and their evaluation. It 
may be used to study function definition, function application, parameter passing 
and recursion. 

Any computable function can be expressed and evaluated using lambda calculus 
or Turing machines, as these are equivalent formalisms. Lambda calculus uses a 
small set of transformation rules, and these include: 


— Alpha-conversion rule (a-conversion)!*; 
. . | 

— Beta-reduction rule (B-reduction)! : 

— Eta-conversion (n-conversion).'* 


Every expression in the 4-calculus stands for a function with a single argument. 
The argument of the function is itself a function with a single argument, and so on. 
The definition of a function is anonymous in the calculus. For example, the function 
that adds one to its argument is usually defined as f(x) = x + 1. However, in i- 
calculus the function is defined as: 


succAAx.x+1. 


The name of the formal argument x is irrelevant and an equivalent definition of 
the function is 4 z. z+ 1. The evaluation of a function f with respect to an argument 
(e.g. 3) is usually expressed by f(3). In A-calculus this would be written as 
(Ax.x + 1)3, and this evaluates to 3 + 1 = 4. Function application is left associa- 
tive: ie. f xy = (f x)y. A function of two variables is expressed in lambda calculus 
as a function of one argument, which returns a function of one argument. This is 
known as currying: e.g. the function f(x, y) = x+y is written as Ax. Ay.x + y. This 
is often abbreviated to Axy.x+y. 

A-calculus is a simple mathematical system, and its syntax is defined as follows: 


<exp> :: = < identifier> | 

i <identifier > . <exp> | —abstraction 
<exp> <exp> | —application 
(<exp>) 


4-Calculus’s four lines of syntax plus conversion rules are sufficient to define 
Booleans, integers, data structures and computations on them. It inspired Lisp and 
modern functional programming languages. The original calculus was untyped, but 
typed lambda calculi have been introduced in recent years. The typed lambda 


This essentially expresses that the names of bound variables is unimportant. 

'5This essentially expresses the idea of function application. 

'“This essentially expresses the idea that two functions are equal if and only if they give the same 
results for all arguments. 
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calculus allows the sets to which the function arguments apply to be specified. For 
example, the definition of the plus function is given as 


plus_Aa,b:N.at+b. 


The lambda calculus makes it possible to express properties of the function 
without reference to members of the base sets on which the function operates. It 
allows functional operations such as function composition to be applied, and one 
key benefit is that the calculus provides powerful support for higher order functions. 
This is important in the expression of the denotational semantics of the constructs of 
programming languages. 


12.6 Lattices and Order 


This section considers some the mathematical structures used in the definition of the 
semantic domains used in denotational semantics. These mathematical structures 
may also be employed to give a secure foundation for recursion (discussed in 
Chap. 4), and it is essential that the conditions in which recursion may be used 
safely be understood. 

It is natural to ask when presented with a recursive definition whether it means 
anything at all, and in some cases the answer is negative. Recursive definitions are a 
powerful and elegant way of giving the denotational semantics of language con- 
structs. The mathematical structures considered in this section include partial 
orders, total orders, lattices, complete lattices and complete partial orders. 


12.6.1 Partially Ordered Sets 


A partial order < ona set P is a binary relation such that for all x, y, z € P the 
following properties hold: 


G) x < x (Reflexivity) 
Gi) x < yandy<x =x=y (Anti-symmetry) 
@ii) x < yandy<z =x<z (Transitivity) 


A set P with an order relation < is said to be a partially ordered set (Fig. 12.5): 


Example 12.4 Consider the power set PX, which consists of all the subsets of the 
set X with the ordering defined by set inclusion. That is, A < B if and only if ACB 
then C is a partial order on PX. 
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Fig. 12.5 Pictorial 
represenation of a partial 
order 


A partially ordered set is a totally ordered set (also called chain) if for all x, 
y € P then either x < y or y < x. That is, any two elements of P are directly 
comparable. 

A partially ordered set P is an anti-chain if for any x, yin P then x < y only if 
x = y. That is, the only elements in P that are comparable to a particular element are 
the element itself. 


Maps between Ordered Sets 


Let P and Q be partially ordered sets then a map ¢ from P to Q may preserve the 
order in P and Q. We distinguish between order preserving, order embedding and 
order isomorphism. These terms are defined as follows: 

Order Preserving (or Monotonic Increasing Function). 

A mapping  : P — Q is said to be order preserving if 


x<y => O(x) < oy). 


Order Embedding 

A mapping  : P — Q is said to be an order embedding if. 

x < yin P if and only ifo(x) < o(y) in Q. 

Order Isomorphism 

The mapping o : P — Q is an order isomorphism if and only if it is an order 
embedding mapping onto Q. 


Dual of a Partially Ordered Set 


The dual of a partially ordered set P (denoted P?) is a new partially ordered set 
formed from P where x <_y holds in P? if and only ify < x holds in P (ie. P? 
is obtained by reversing the order on P). 

For each statement about P there is a corresponding statement about P?. Given 
any statement ® about a partially ordered set, then the dual statement ©? is obtained 
by replacing each occurrence of < by > and vice versa. 


Duality Principle 


Given that statement ® is true of a partially ordered set P, then the statement ©? is 
true of P?. 
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Maximal and Minimum Elements 


Let P be a partially ordered set and let QCP then 


(i) a € QO isa maximal element of O ifa<xE€O>a=x. 
(ii) a € Qis the greatest (or maximum) element of Q if a > x for every x € Q, 
and in that case we write a = max Q. 


A minimal element of Q and the least (or minimum) are defined dually by 
reversing the order. The greatest element (if it exists) is called the top element and is 
denoted by T. The least element (if it exists) is called the bottom element and is 
denoted by L. 


Example 12.5 Let X be a set and consider PX the set of all subsets of X with the 
ordering defined by set inclusion. The top element T is given by X, and the bottom 
element | is given by 9). 


A finite totally ordered set always has top and bottom elements, but an infinite 
chain need not have. 


12.6.2 Lattices 


Let P be a partially ordered set and let SCP. An element x € P is an upper bound of 
S ifs < x for all s € S. A lower bound is defined similarly. 

The set of all upper bounds for S is denoted by S", and the set of all lower bounds 
for S is denoted by S’. 


SY = {x € P| (Vs € S)s <x}, 
S' = {x € P| (Vs € S)s>x}. 


If S“ has a least element x then x is called the least upper bound of S. Similarly, if 
S' has a greatest element x then x is called the greatest lower bound of S. 
Another words, x is the least upper bound of S if. 


(i) x is an upper bound of S. 
(ii) x < y for all upper bounds y of S. 


The least upper bound of S is also called the supremum of S denoted (sup S), and 
the greatest lower bound is also called the infimum of S, and is denoted by inf S. 
Join and Meet Operations 


The join of x and y (denoted by x V y) is given by sup{x, y} when it exists. The meet 
of x and y (denoted by x A y) is given by inf{x, y} when it exists. 
The supremum of S$ is denoted by \/S, and the infimum of S is denoted by /\S. 
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Definition 
Let P be a non-empty partially ordered set then. 


@) Ifx V y and x A y exist for all x, y € P then P is called a lattice. 
(ii) If VS and AS exist for all SCP then P is called a complete lattice 


Every non-empty finite subset of a lattice has a meet and a join (inductive 
argument can be used), and every finite lattice is a complete lattice. Further, any 


complete lattice is bounded—i.e. it has top and bottom elements (Fig. 12.6). 


Example 12.6 Let X be a set and consider PX the set of all subsets of X with the 
ordering defined by set inclusion. Then PX is a complete lattice in which. 


v{Ajli € I} = UA; 
A{Ajli € I} = NAj. 


Consider the set of natural numbers N and consider the usual ordering of < . 
Then N is a lattice with the join and meet operations defined as 


x V y = max(x, y) 
x Ay = min(x, y). 


Another possible definition of the meet and join operations are in terms of the 
greatest common multiple and lease common divisor. 


xV y = Iem(x, y) 
xAy = ged(x, y). 
Fig. 12.6 Pictorial 


represenation of a complete 
lattice 
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12.6.3 Complete Partial Orders 


Let S be a non-empty subset of a partially ordered set P. Then 


Gj) S is said to be a directed set if for every finite subset F of S there exists 
z € S such that z € F". 

(ii) S is said to be consistent if for every finite subset F of S there exists z € P such 
that z € F". 


A partially ordered set P is a complete partial order (CPO) if 


(i) P has a bottom element L. 
(ii) \VD exists for each directed subset D of P. 


The simplest example of a directed set is a chain, and we note that any complete 
lattice is a complete partial order, and that any finite lattice is a complete lattice. 


12.6.4 Recursion 


Recursive definitions arise frequently in programs and offer an elegant way to 
define routines and data types. A recursive routine contains a direct or indirect call 
to itself, and a recursive data type contains a direct or indirect reference to speci- 
mens of the same type. Recursion needs to be used with care, as there is always a 
danger that the recursive definition may be circular (i.e. defines nothing). It is 
therefore important to investigate when a recursive definition may be used safely, 
and to give a mathematical definition of recursion. 

The control flow in a recursive routine must contain at least one non-recursive 
branch since if all possible branches included a recursive form the routine could 
never terminate. The value of at least one argument in the recursive call is different 
from the initial value of the formal argument as otherwise the recursive call would 
result in the same sequence of events and therefore would never terminate. 

The mathematical meaning of recursion is defined in terms of fixed point theory, 
which is concerned with determining solutions to equations of the form x= t(x), 
where the function t is of the form t: X — X. 

A recursive definition may be interpreted as a fixpoint equation of the form 
f = Of); i.e. the fixpoint of a high-level functional ® that takes a function as an 
argument. For example, consider the functional ® defined as follows: 


MA Jf An - ifn = Othen | else n*f(n — 1). 
Then a fixpoint of ® is a function f such that f = ®(f) or another words 


f =4n- ifn = Othen 1 else n*f(n — 1). 
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Clearly, the factorial function is a fixpoint of ®, and it is the only total function 
that is a fixpoint. The solution of the equation f = ®(f) (where ® has a fixpoint) is 
determined as the limit f of the sequence of functions fo, fi, fo, ...., where the f; are 
defined inductively as 


foAd (the empty partial function) 
fi AP(F-1). 


Each f; may be viewed as a successive approximation to the true solution f of the 
fixpoint equation, with each f; bringing a little more information on the solution 
than its predecessor fj.1. 

The function f; is defined for one more value than f;_;, and gives the same result 
for any value for which they are both defined. The definition of the factorial 
function is thus built up as follows: 


fo Ad (the empty partial function) 
FrA{O— 1} 

fA{O>11 31} 
fA{— 1,15 1,2 — 2} 

PAIS 1,1 3125936} 


For every i, the domain of f; is the interval 1, 2, ... i— 1 and f; (n) = n! for any 
n in this interval. Another words f; is the factorial function restricted to the interval 
1, 2, ... i— 1. The sequence of f; may be viewed as successive approximations of 
the true solution of the fixpoint equation (which is the factorial function), with each 
j, bringing defined for one more value that its predecessor f;_;, and defining the same 
result for any value for which they are both defined. 

The candidate fixpoint f,, is the limit of the sequence of functions f;, and is the 
union of all the elements in the sequence. It may be written as follows: 


foo AVDU®(0) U®(O(9)) U... = Uinfi 


where the sequence f; is defined inductively as 


fo Ad (the empty partial function) 
fi+1 AfiU Of). 
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This forms a subset chain where each element is a subset of the next, and it 
follows by induction that 


Fis1=U po. OG). 


A general technique for solving fixpoint equations of the form / = t(/) for some 
functional t is to start with the least defined function and iterate with t. The union 
of all the functions obtained as successive sequence elements is the fixpoint. 

The conditions in which f,, is a fixpoint of ® is the requirement for ®(f,,) = fr. 
This is equivalent to 


O( U ;-nfi) == Uinfi 
®(U jnfi) = Uin®(f). 


A sufficient condition for ® to have a fixpoint is that the property ®( U;.nf;) = 
Ujn®(f;) holds for any subset chain f;. 

We discussed recursion earlier in Chap. 4, and a more detailed account on the 
mathematics of recursion is in Chap. 8 of [7]. 


12.7 Review Questions 


. Explain the difference between syntax and semantics. 
. Describe the Chomsky hierarchy of grammars and give examples of each 
type. 

3. Show that a grammar may be ambiguous leading to two difference parse 
trees. What problems does this create and how should it be dealt with? 

4. Describe axiomatic semantics, operation semantics and denotational 
semantics and explain the differences between them. 

5. Explain partial orders, lattices and complete partial orders. Give examples 
of each. 

6. Show how the meaning of recursion is defined with fixpoint theory. 


Ne 


12.8 Summary 


This chapter considered two key parts to any programming language, namely 
syntax and semantics. The syntax of the language is concerned with the production 
of grammatically correct programs in the language, whereas the semantics of the 
language is deeper and is concerned with the meaning of what has been written by 
the programmer. 
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There are several approaches to defining the semantics of programming lan- 
guages, and these include axiomatic, operational and denotational semantics. 
Axiomatic semantics is concerned with defining properties of the language in terms 
of axioms; operational semantics is concerned with defining the meaning of the 
language in terms of an interpreter; and denotational semantics is concerned with 
defining the meaning of the phrases in a language by the denotation or mathe- 
matical meaning of the phrase. 

Compilers are programs that translate a program that is written in some pro- 
gramming language into another form. It involves syntax analysis and parsing to 
check the syntactic validity of the program; semantic analysis to determine what the 
program should do; optimization to improve the speed and performance of the 
compiler and code generation in some target language. 

Various mathematical structures including partial orders, total orders, lattices 
and complete partial orders were considered. These are useful in the definition of 
the denotational semantics of a language, and in giving a mathematical interpre- 
tation of recursion. 
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13.1. Introduction 


It is impossible for a human or machine to write out all of the members of an 
infinite countable set, such as the set of natural numbers N. However, humans can 
do something quite useful in the case of certain enumerable infinite sets: they can 
give explicit instructions (that may be followed by a machine or another human) to 
produce the nth member of the set for an arbitrary finite n. The problem remains 
that for all but a finite number of values of n it will be physically impossible for any 
human or machine to actually carry out the computation, due to the limitations on 
the time available for computation, the speed at which the individual steps in the 
computation may be carried out, and due to finite materials. 

The intuitive meaning of computability is in terms of an algorithm (or effective 
procedure) that specifies a set of instructions to be followed to complete the task. 
Another words, a function fis computable if there exists an algorithm that produces 
the value of f correctly for each possible argument of f. The computation of ffor a 
particular argument x just involves following the instructions in the algorithm, and 
it produces the result f(x) in a finite number of steps if x is in the domain of f. If x is 
not in the domain of f then the algorithm may produce an answer saying so or it 
might run forever never halting. A computer program implements an algorithm. 

The concept of computability may be made precise in several equivalent ways 
such as Church’s lambda calculus, recursive function theory or by the theoretical 
Turing machines.' These are all equivalent and perhaps the most well known is the 
Turing machine (discussed in Chap. 7). This is a mathematical machine with a 
potentially infinite tape divided into frames (or cells) in which very basic operations 


'The Church-Turing thesis states that anything that is computable is computable by a Turing 
Machine. 
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can be carried out. The set of functions that are computable are those that are 
computable by a Turing machine. 

Decidability is an an important topic in contemporary mathematics. Church and 
Turing independently showed in 1936 that mathematics is not decidable. In other 
words, there is no mechanical procedure (i.e. algorithm) to determine whether an 
arbitrary mathematical proposition is true or false, and so the only way is to 
determine the truth or falsity of a statement is try to solve the problem. The fact that 
there is no a general method to solve all instances of a specific problem, as well as 
the impossibility of proving or disproving certain statements within a formal system 
may suggest limitations to human and machine knowledge. 


13.2 Logicism and Formalism 


Gottlob Frege (Fig. 14.8) was a nineteenth-century German mathematician and 
logician who invented a formal system which is the basis of modern predicate logic. 
It included axioms, definitions, universal and existential quantification, and for- 
malization of proof. His objective was to show that mathematics was reducible to 
logic (logicism) but his project failed as one of the axioms used as part of the 
axiomatisation of set theory led to inconsistentcy. 

This inconsistency was pointed out by Bertrand Russell, and it is known as 
Russell's paradox.” Russell later introduced the theory of types to deal with the 
paradox, and he jointly published Principia Mathematica with Whitehead as an 
attempt to derive the truths of arithmetic from a set of logical axioms and rules of 
inference. 

The sentences of Frege’s logical system denote the truth-values of true or false. 
The sentences may include expressions such as equality (x = y), and this returns 
true if x is the same as y, and false otherwise. Similarly, a more complex expression 
such as f(x, y, Z) = w is true if f(x, y, z) is identical with w, and false otherwise. Frege 
represented statements such as ‘5 is a prime’ by ‘P(5)’ where P() is termed a 
concept. The statement P(x) returns true if x is prime and false otherwise. His 
approach was to represent a predicate as a function of one variable which returns a 
Boolean value of true or false. 

Formalism was proposed by Hilbert (Fig. 13.1) as a foundation for mathematics 
in the early twentiety century. The motivation for the programme was to provide a 
secure foundations for mathematics, and to resolve the contradictions in the for- 
malization of set theory identified by Russell’s paradox. The presence of a con- 
tradiction in a theory means the collapse of the whole theory, and so it was seen as 
essential that there be a proof of the consistency of the formal system. The formalist 
approach means that the methods of proof in mathematics are formalized with 
axioms and rules of inference. 


*Russell’s paradox (discussed in Chap. 2) considers the question as to whether the set of all sets 
that contain themselves as members is a set. In either case there is a contradiction. 
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Fig. 13.1 David Hilbert 


Formalism is a formal system that contains meaningless symbols together with 
rules for manipulating them. The individual formulas are certain finite sequences of 
symbols obeying the syntactic rules of the formal language. A formal system 
consists of: 


— A formal language; 
— A set of axioms; 
— Rules of inference. 


The expressions in a formal system are terms, and a term may be simple or 
complex. A simple term may be an object such as a number, and a complex term 
may be an arithmetic expression such as 4° + 1. A complex term is formed via 
functions, and the expression above uses two functions namely the cube function 
with argument 4 and the plus function with two arguments. 

A formal system is generally intended to represent some aspect of the real world. 
A rule of inference relates a set of formulas (P1, Po, ... P;) called the premises to the 
consequence formula P called the conclusion. For each rule of inference there is a 
finte procedure for determining whether a given formula Q is an immediate con- 
sequence of the rule from the given formulas (P;, P2, ... P,). A proof in a formal 
system consists of a finite sequence of formulae, where each formula is either an 
axiom or derived from one or more preceding formulae in the sequence by one of 
the rules of inference. 

Hilbert’s programme was concerened with the formalization of mathematics (i.e. 
the axiomatization of mathematics) together with a proof that the axiomatization 
was consistent (i.e. there is no formula A such that both A and A are deducible in 
the calculus). The specific objectives of Hilbert’s programme were to 


— Provide a formalism of mathematics. 
— Show that the formalization of mathematics was complete: i.e. all mathematical 
truths can be proved in the formal system. 
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— Provide a proof that the formal system is is consistent (i.e. that no contradictions 
may be derived). 

— Show that mathematics is decidable: i.e. there is an algorithm to determine the 
truth of falsity of any mathematical statement. 


The formalist movement in mathematics led to the formalization of large parts of 
mathematics, where theorems could be proved using just a few mechanical rules. 
The two most comprehensive formal systems developed were Principia Mathe- 
matica by Russell and Whitehead, and the axiomatization of set theory by 
Zermelo-Fraenkel (subsequently developed further by von Neumann). 

Principia Mathematica is a comprehensive three volume work on the logical 
foundations of mathematics written by Bertrand Russel and Alfred Whitehead 
between 1910 and 1913. Its goal was to show that all of the concepts of mathe- 
matics can be expressed in logic, and that all of the theorems of mathematics can be 
proved using only the logical axioms and rules of inference of logic. It covered set 
theory, ordinal numbers and real numbers, and it showed that in principle that large 
parts of mathematics could be developed using logicism [1]. 

It avoided the problems with contradictions that arose with Frege’s system by 
introducing the theory of types in the system. The theory of types meant that one 
could no longer speak of the set of all sets, as a set of elements is of a different type 
from that of each of its elements, and so Russell’s paradox was avoided. It remained 
an open question at the time as to whether the Principia was consistent and 
complete. That is, is it possible to derive all the truths of arithmetic in the system 
and is it possible to derive a contradiction from the Principia’s axioms? However, it 
was clear from the three volume work that the development of mathematics using 
the approach of the Principia was extremely lengthy and time consuming. 

Chapter 17 discusses early automation of mathematical proof in the 1950s, 
including the Logic Theorist (LT) computer program that was demonstrated at the 
Dartmounth conference on Artificial Intelligence in 1956, as well as interactive and 
automated theorem provers. LT was developed by Allen Newell and Herbert 
Simon, and it could prove 38 of the first 52 theorems from Principia Mathematica. 


13.3 Decidability 


The question remained whether these axioms and rules of inference are sufficient to 
decide any mathematical question that can be expressed in these systems. Hilbert 
believed that every mathematical problem could be solved, and that the truth or 
falsity of any mathematical proposition could be determined in a finite number of 
steps. He outlined twenty-three key problems at the International Congress of 
Mathematics in Paris in 1900 that needed to be solved by mathematicians in the 
twentieth century. 
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He believed that the formalism of mathematics would allow a mechanical pro- 
cedure (or algorithm) to determine whether a particular statement was true or false. 
The problem of the decidability of mathematics is known as the decision problem 
(Entscheidungsproblem). 

The question of the decidability of mathematics had been considered by Leibnitz 
in the seventeenth century. He had constructed a mechanical calculating machine, 
and wondered if a machine could be built that could determine whether particular 
mathematical statements are true or false. 


Definition 13.1 (Decidability) 
Mathematics is decidable if the truth or falisty of any mathematical proposition may 
be determined by an algorithm. 

Church and Turing independently showed this to be impossible in 1936. Church 
developed the lambda calculus in the 1930s as a tool to study computability,’ and 
he showed that anything that is computable is computable by the lambda calculus. 
Turing showed that decidability was related to the halting problem for Turing 
machines, and that therefore if first-order logic were decidable then the halting 
problem for Turing machines could be solved. However, he had already proved that 
there was no general algorithm to determine whether a given Turing machine halts 
or not. Therefore, first-order logic is undecidable. 

The question as to whether a given Turing machine halts or not can be for- 
mulated as a first-order statement. If a general decision procedure exists for 
first-order logic, then the statement of whether a given Turing machine halts or not 
is within the scope of the decision algorithm. However, Turing had already proved 
that the halting problem for Turing machines is not computable: i.e. it is not 
possible algorithmically to decide whether or not any given Turing machine will 
halt or not. Therefore, since there is no general algorithm that can decide whether 
any given Turing machine halts, there is no general decision procedure for 
first-order logic. The only way to determine whether a statement is true or false is to 
try to solve it. However, if one tries but does not succeed this does not prove that an 
answer does not exist. 

There are first-order theories that are decidable. However, first-order logic that 
includes Peano’s axioms of arithmetic (or any formal system that includes addition 
and multiplication) cannot be decided by an algorithm. That is, there is no algorithm 
to determine whether an arbitrary mathematical proposition is true or false. 
Propositional logic is decidable as there is a procedure (e.g. using a truth table) to 
determine whether an arbitrary formula is valid* in the calculus. 


>The Church Turing Thesis states that anytime that is computable is computable by Lambda 
Calculus or equivalently by a Turing Machine. 

“A well-formed formula is valid if it follows from the axioms of first-order logic. A formula is 
valid if and only if it is true in every interpretation of the formula in the model. 
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Fig. 13.2 Kurt Gédel 


Gédel (Fig. 13.2) proved that first-order predicate calculus is complete. i.e. all 
truths in the predicate calculus can be proved in the language of the calculus. 


Definition 13.2 (Completeness) 

A formal system is complete if all the truths in the system can be derived from 
the axioms and rules of inference. 

Gédel’s first incompleteness theorem showed that first-order arithmetic is 
incomplete; i.e. there are truths in first-order arithmetic that cannot be proved in the 
language of the axiomatization of first-order arithmetic. Gddel’s second incom- 
pleteness theorem showed that that any formal system extending basic arithmetic 
cannot prove its own consistency within the formal system. 


Definition 13.3 
(Consistency) A formal system is consistent if there is no formula A such that A 
and ~A are provable in the system (i.e. there are no contradictions in the system). 


13.4 Computability 


Alonzo Church (Fig. 13.3) developed the lambda calculus in the mid 1930s, as part of 
his work into the foundations of mathemtics. Turing published a key paper on com- 
putability in 1936, which introduced the theoretical machine known as the Turing 
machine. This machine is computationally equivalent to the lambda calculus, and is 
capable of performing any conceivable mathematical problem that has an algorithm. 


Definition 13.4 (Algorithm) 

An algorithm (or effective procedure) is a finite set of unambiguous instructions 
to perform a specific task. 

A function is computable if there is an effective procedure or algorithm to 
compute f for each value of its domain. The algorithm is finite in length and 
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Fig. 13.3 Alonzo Church 


sufficiently detailed so that a person can execute the instructions in the algorithm. 
The execution of the algorithm will halt in a finite number of steps to produce the 
value of f(x) for all x in the domain of f, However, if x is not in the domain of f then 
the algorithm may produce an answer saying so, or it may get stuck, or it may run 
forever never halting. 

The Church-Turing Thesis that states that any computable function may be 
computed by a Turing machine. There is overwhelming evidence in support in 
support of this thesis, including the fact that alternative formalizations of com- 
putability in terms of lambda calculus, recursive function theory, and Post systems 
have all been shown to be equivalent to Turing machines. 

A Turing machine (discussed previously in Chap. 7) consists of a head and a 
potentially infinite tape that is divided into cells. Each cell on the tape may be either 
blank or printed with a symbol from a finite alphabet of symbols. The input tape 
may initially be blank or have a finite number of cells containing symbols. 

At any step, the head can read the contents of a frame. The head may erase a 
symbol on the tape, leave it unchanged, or replace it with another symbol. It may 
then move one position to the right, one position to the left, or not at all. If the frame 
is blank, the head can either leave the frame blank or print one of the symbols. 

Turing believed that a human with finite equipment and with an unlimited 
supply of paper could do every calculation. The unlimited supply of paper is 
formalized in the Turing machine by a tape marked off in cells. 

We gave a formal definition of a Turing machine as a 7-tuple M = (Q,T, b, &, 6, 
qo, F) in Chap. 7. We noted that the Turing machine is a simple theoretical 
machine, but it is equivalent to an actual physical computer in the sense that they 
both compute exactly the same set of functions. A Turing machine is easier to 
analyze and prove things about than a real computer. 

A Turing machine is essentially a finite state machine (FSM) with an unbounded 
tape. The machine may read from and write to the tape and the tape provides 
memory and acts as the store. The finite state machine is essentially the control unit 
of the machine, whereas the tape is a potentially infinite and unbounded store. 
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A real computer has a large but finite store, whereas the store in a Turing machine is 
potentially infinite. However, the store in a real computer may be extended with 
backing tapes and disks, and in a sense may be regarded as unbounded. The 
maximum amount of tape that may be read or written within n steps is n. 

A Turing machine has an associated set of rules that defines its behaviour. These 
tules are defined by the transition function that specify the actions that a machine 
will perform with respect to a particular input. The behaviour will depend on the 
current state of the machine and the contents of the tape. 

A Turing machine may be programmed to solve any problem for which there is 
an algorithm. However, if the problem is unsolvable then the machine will either 
stop in a non-accepting state or compute forever. The solvability of a problem may 
not be determined beforehand, but, there is, of course, some answer (i.e. either the 
machine either halts or it computes forever). 

Turing showed that there was no solution to the decision problem (Entschei- 
dungsproblem) posed by Hilbert. Hilbert believed that the truth or falsity of a 
mathematical problem may always be determined by a mechanical procedure, and 
he believed that first-order logic is decidable: i.e. there is a decision procedure to 
determine if an arbitrary formula is a theorem of the logical system. 

Turing was skeptical on the decidability of first-order logic, and the Turing 
machine played a key role in refuting Hilbert’s claim of the decidability of 
first-order logic. 

The question as to whether a given Turing machine halts or not can be for- 
mulated as a first-order statement. If a general decision procedure exists for 
first-order logic, then the statement of whether a given Turing machine halts or not 
is within the scope of the decision algorithm. However, Turing had already proved 
that the halting problem for Turing machines is not computable: i.e. it is not 
possible algorithmically to decide whether a given Turing machine will halt or not. 
Therefore, there is no general algorithm that can decide whether a given Turing 
machine halts. In other words, there is no general decision procedure for first-order 
logic. The only way to determine whether a statement is true or false is to try to 
solve it. 

Turing also introduced the concept of a Universal Turing Machine, and this 
machine is able to simulate any other Turing machine. Turing’s results on com- 
putability were proved independently of Church’s lambda calculus equivalent 
results in computability. Turing’s studied at Priceton Univsersity in 1937 and 1938 
and was awarded a PhD from the university in 1938. His research supervisor was 
Alonzo Church.° 


Alonzo Church was a famous American mathematician and logician who developed the lambda 
calculus. He also showed that Peano arithmetic and first-order logic were undecidable. Lambda 
calculus is equivalent to Turing machines and whatever may be computed is computable by 
Lambda calculus or a Turing machine. 
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Question 13.1 (Halting Problem) 

Given an arbitrary program is there an algorithm to decide whether the pro- 
gram will finish running or will continue running forever? Another words, given a 
program and an input will the program eventually halt and produce an output or 
will it run forever? 


Note (Halting Problem) 

The halting problem was one of the first problems that was shown to be 
undecidable: i.e. there is no general decision procedure or algorithm that may be 
applied to an arbitrary program and input to decide whether the program halts or not 
when run with that input. 


Proof We assume that there is an algorithm (i.e. a computable function function H 
(i, j)) that takes any program i (program i refers to the ith program in the enu- 
meration of all the programs) and arbitrary input j to the program such that 


. . _ J | Ifprogram halts on input). 
Hibsi) = c otherwise 


We then employ a diagonalization argument® to show that every computable 
total function f with two arguments differs from the desired function H. First, we 
construct a partial function g from any computable function f with two arguments 
such that g is computable by some program e. 


Poe {t if f(i,i) =0 


undefined otherwise. 


There is a program e that computes g, and this program is one of the programs in 
which the halting problem is defined. One of the following two cases must hold: 


ge) =f(e,e) =0. (13.1) 
In this case H(e, e) = 1 because e halts on input e. 
g(e) is undefined andf(e, e) 4 0. (13.2) 


In this case H(e, e) = 0 because the program e does not halt on input e. 


This is similar to Cantor’s diagonalization argument that shows that the Real numbers are 
uncountable. This argument assumes that it is possible to enumerate all real numbers between 0 
and 1, and it then constructs a number whose nth decimal differs from the nth decimal position in 
the nth number in the enumeration. If this holds for all n then the newly defined number is not 
among the enumerated numbers. 
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In either case, fis not the same function as H. Further, since f was an arbitrary 
total computable function all such functions must differ from H. Hence, the function 
H is not computable and there is no such algorithm to determine whether an 
arbitrary Turing machine halts for an input x. Therefore, the halting problem is not 
decidable. 


13.5 Computational Complexity 


An algorithm is of little practical use if it takes millions of years to compute 
particular instances. There is a need to consider the efficiency of the algorithm due 
to practical considerations. Chapter 10 discussed cryptography and the RSA 
algorithm, and the security of the RSA encryption algorithm is due to the fact that 
there is no known efficient algorithm to determine the prime factors of a large 
number. 

There are often slow and fast algorithms for the same problem, and a measure of 
complexity is the number of steps in a computation. An algorithm is of time 
complexity f(n) if for all n and all inputs of length n the execution of the algorithm 
takes at most f(7) steps. 

An algorithm is said to be polynomially bounded if there is a polynomial p 
(n) such that for all n and all inputs of length n the execution of the algorithm takes 
at most p(n) steps. The notation P is used for all problems that can be solved in 
polynomial time. 

A problem is said to be computationally intractable if it may not be solved in 
polynomial time—that is, there is no known algorithm to solve the problem in 
polynomial time. 

A problem L is said to be in the set NP (non-deterministic polynomial time 
problems) if any given solution to L can be verified quickly in polynomial time. 
A non-deterministic Turing machine may have several possibilities for its beha- 
viour, and an input may give rise to several computations. 

A problem is NP complete if it is in the set NP of non-deterministic polynomial 
time problems, and it is also in the class of NP hard problems. A key characteristic 
to NP complete problems is that there is no known fast solution to them, and the 
time required to solve the problem using known algorithms increases quickly as the 
size of the problem grows. Often, the time required to solve the problem is in 
billions or trillions of years. Although any given solution can be verified quickly 
there is no known efficient way to find a solution. 
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13.6 Review Questions 


1. Explain computability and decidability. 

2. What were the goals of logicism and formalism and how successful were 
these movement in mathematics? 

. What is a formal system? 

4. Explain the difference between consistency, completeness and 
decidability. 

. Describe a Turing machine and explain its significance in computability. 

. Describe the halting problem and show that it is undecidable. 

7. Discuss the complexity of an algorithm and explain terms such as 

“polynomial bounded’, ‘computationally intractable’ and ‘NP complete’. 


1S) 


nN 


13.7. Summary 


This chapter provided an introduction to computability and decidability. The 
intuitive meaning of computability is that in terms of an algorithm (or effective 
procedure) that specifies a set of instructions to be followed to solve the problem. 
Another words, a function fis computable if there exists an algorithm that produces 
the value of f correctly for each possible argument of f. The computation of ffor a 
particular argument x just involves following the instructions in the algorithm, and 
it produces the result f(x) in a finite number of steps if x is in the domain of f. 

The concept of computability may be made precise in several equivalent ways 
such as Church’s lambda calculus, recursive function theory or by the theoretical 
Turing machines. The Turing machine is a mathematical machine with a potentially 
infinite tape divided into frames (or cells) in which very basic operations can be 
carried out. The set of functions that are computable are those that are computable 
by a Turing machine. 

A formal system contains meaningless symbols together with rules for manip- 
ulating them, and is generally intended to represent some aspect of the real world. 
The individual formulas are certain finite sequences of symbols obeying the syn- 
tactic rules of the formal language. A formal system consists of a formal language, a 
set of axioms and rules of inference. 

Church and Turing independently showed in 1936 that mathematics is not 
decidable. In other words, it is not possible to determine the truth or falisty of any 
mathematical proposition by an algorithm. 

Turing had already proved that the halting problem for Turing machines is not 
computable: i.e. it is not possible algorithmically to decide whether a given Turing 
machine will halt or not. He then applied this result to first-order logic to show that 
it is undecidable. That is, the only way to determine whether a statement is true or 
false is to try to solve it. 
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The complexity of an algorithm was discussed, and it was noted that an algo- 
rithm is of little practical use if it takes millions of years to compute the solution. 
There is a need to consider the efficiency of the algorithm due to practical con- 
siderations. The class of polynomial time bound problems and non-deterministic 
polynomial time problems were considered, and it was noted that the security of 
various cryptographic algorithms is due to the fact that there are no time efficient 
algorithms to determine the prime factors of large integers. 

The reader is referred to [2] for a more detailed account of decidability and 
computability. 
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14.1. Introduction 


Logic is concerned with reasoning and with establishing the validity of arguments. 
It allows conclusions to be deduced from premises according to logical rules, and 
the logical argument establishes the truth of the conclusion provided that the pre- 
mises are true. 

The origins of logic are with the Greeks who were interested in the nature of 
truth. The sophists (e.g. Protagoras and Gorgias) were teachers of rhetoric, who 
taught their pupils techniques in winning an argument and convincing an audience. 
Plato explores the nature of truth in some of his dialogues, and he is critical of the 
position of the sophists who argue that there is no absolute truth, and that truth 
instead is always relative to some frame of reference. The classic sophist position is 
stated by Protagoras “Man is the measure of all things: of things which are, that 
they are, and of things which are not, that they are not.” In other words, what is true 
for you is true for you, and what is true for me is true for me. 

Socrates had a reputation for demolishing an opponent's position, and the 
Socratean enquiry consisted of questions and answers in which the opponent would 
be led to a conclusion incompatible with his original position. The approach was 
similar to a reductio ad absurdum argument, although Socrates was a moral 
philosopher who did no theoretical work on logic. 

Aristotle did important work on logic, and he developed a system of logic, 
syllogistic logic, that remained in use up to the nineteenth century. Syllogistic logic 
is a ‘term logic’, with letters used to stand for the individual terms. A syllogism 
consists of two premises and a conclusion, where the conclusion is a valid 
deduction from the two premises. Aristotle also did some early work on modal logic 
and was the founder of the field. 

The Stoics developed an early form of propositional logic, where the assertibles 
(propositions) have a truth-value such that at any time they are either true or false. 
The assertibles may be simple or non-simple, and various connectives such as 
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conjunctions, disjunctions and implications are used in forming more complex 
assertibles. 

George Boole developed his symbolic logic in the mid-1800s, and it later formed 
the foundation for digital computing. Boole argued that logic should be considered 
as a separate branch of mathematics, rather than a part of philosophy. He argued 
that there are mathematical laws to express the operation of reasoning in the human 
mind, and he showed how Aristotle’s syllogistic logic could be reduced to a set of 
algebraic equations. 

Logic plays a key role in reasoning and deduction in mathematics, but it is 
considered a separate discipline to mathematics. There were attempts in the early 
twentieth century to show that all mathematics can be derived from formal logic 
and that the formal system of mathematics would be complete, with all the truths of 
mathematics provable in the system (see Chap. 13). However, this program failed 
when the Austrian logician, Kurt Goedel, showed that there are truths in the formal 
system of arithmetic that cannot be proved within the system (i.e. first-order 
arithmetic is incomplete). 


14.2 Syllogistic Logic 


Early work on logic was done by Aristotle in the fourth century B.C. in the 
Organon [1]. Aristotle regarded logic as a useful tool of enquiry into any subject, 
and he developed syllogistic logic. This is a form of reasoning in which a con- 
clusion is drawn from two premises, where each premise is in a subject-predicate 
form. A common or middle term is present in each of the two premises but not in 
the conclusion. For example 

All Greeks are mortal. 


Socrates is a Greek 


Therefore Socrates is mortal 


The common (or middle) term in this example is ‘Greek’. It occurs in both 
premises but not in the conclusion. The above argument is valid, and Aristotle 
studied and classified the various types of syllogistic arguments to determine those 
that were valid or invalid. Each premise contains a subject and a predicate, and the 
middle term may act as a subject or a predicate. Each premise is a positive or 
negative affirmation, and an affirmation may be universal or particular. The uni- 
versal and particular affirmations and negatives are described in the table below 
(Table 14.1) 
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Table 14.1 Types of 


—s ! Type Symbol Example 

pilceste penke Universal GAM All Greeks are mortal 
affirmative 
Universal negative GEM No Greek is mortal 
Particular GIM Some Greek is mortal 
affirmative 
Particular negative GOM Some Greek is not mortal 

Table 14.2 Forms of Form Form Form Form 

syllogistic premises (i) (ii) (iii) (iv) 
Premise | MP PM PM MP 
Premise 2 MS SM MS SM 
Conclusion SP SP SP SP 


This leads to four basic forms of syllogistic arguments (Table 14.2), where the 
middle is the subject of both premises; the predicate of both premises; and the 
subject of one premise and the predicate of the other premise. 

There are four types of premises (A, E, I and O), and therefore, sixteen sets of 
premise pairs for each of the forms above. However, only some of these premise 
pairs will yield a valid conclusion. Aristotle went through every possible premise 
pair to determine if a valid argument may be derived. The syllogistic argument 
above is of the form (iv) and is valid 


Syllogistic logic is a ‘term logic’ with letters used to stand for the individual 
terms. Syllogistic logic was the first attempt at a science of logic and it remained in 
use up to the nineteenth century. There are many limitations to what it may express, 
and on its suitability as a representation of how the mind works. 


14.3. Paradoxes and Fallacies 


A paradox is a statement that apparently contradicts itself, and it presents a situation 
that appears to defy logic. Some logical paradoxes have a solution, whereas others 
are contradictions or invalid arguments. There are many examples of paradoxes, 
and they often arise due to self-reference in which one or more statements refer to 
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each other. We discuss several paradoxes such as the liar paradox and the sorites 
paradox, which were invented by Eubulides of Miletus, and the barber paradox, 
which was introduced by Russell to explain the contradictions in naive set theory. 

An example of the liar paradox is the statement “Everything that I say is false”, 
which is made by the liar. This looks like a normal sentence but it is also saying 
something about itself as a sentence. If the statement is true, then the statement must 
be false, since the meaning of the sentence is that every statement (including the 
current statement) made by the liar is false. If the current statement is false, then the 
statement that everything that I say is false is false, and so this must be a true 
statement. 

The Epimenides paradox is a variant of the liar paradox. Epimenides was a 
Cretan who allegedly stated “All Cretans are liars”. If the statement is true, then 
since Epimenides is Cretan, he must be a liar, and so the statement is false and we 
have a contradiction. However, if we assume that the statement is false and that 
Epimenides is lying about all Cretan being liars, then we may deduce (without 
contradiction) that there is at least one Cretan who is truthful. So in this case the 
paradox can be avoided. 

The sorites paradox (paradox of the heap) involves a heap of sand in which 
grains are individually removed. It is assumed that removing a single grain of sand 
does not turn a heap into a non-heap, and the paradox is to consider what happens 
when the process is repeated often enough. Is a single remaining grain a heap? 
When does it change from being a heap to a non-heap? This paradox may be 
avoided by specifying a fixed boundary of the number of grains of sand required to 
form a heap, or to define a heap as a collection of multiple grains (> 2 grains). Then 
any collection of grains of sand less than this boundary is not a heap. 

The barber paradox is a variant of Russell’s paradox (a contradiction in naive 
set theory), which was discussed in chapter two. In a village, there is a barber who 
shaves everyone who does not shave himself, and no one else. Who shaves the 
barber? The answer to this question results in a contradiction, as the barber cannot 
shave himself, since he shaves only those who do not shave themselves. Further, as 
the barber does not shave himself, then he falls into the group of people who would 
be shaved by the barber (himself). Therefore, we conclude that there is no such 
barber (or that the barber has a beard). 

The purpose of a debate is to convince an audience of the correctness of your 
position and to challenge and undermine your opponent’s position. Often, the 
arguments made are factual, but occasionally individuals skilled in rhetoric and 
persuasion introduce bad arguments as a way to persuade the audience. Aristotle 
studied and classified bad arguments (known as fallacies), and these include fal- 
lacies such as the ad hominem argument; the appeal to authority argument; and the 
straw man argument. The fallacies are described in more detail in Table 14.3. 
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Table 14.3 Table: fallacies in arguments 


Fallacy 


Hasty/accident 
generalization 
Secundum Quid 


Slippery slope 


Against the person 
Ad Hominem 


Appeal to people 
Ad Populum 


Appeal to authority Ad 
Verecundiam 


Appeal to pity 
Ad Misericordiam 


Appeal to ignorance 
Argumentum ad 
ignoratiam 

Straw man argument 
Ignoratio Elenchi 


Begging the question 
Petitio Principii 


Red herring 


False dichotomy 
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Description/example 


This is a bad argument that involves a generalization that 
disregards exceptions. It involves drawing a conclusion about all 
instances based on a small number of instances 


This argument outlines a chain reaction leading to a highly 
undesirable situation that will occur if a certain situation is 
allowed. The claim is that even if one step is taken onto the 
slippery slope, then we will fall all the way down to the bottom 


The focus of this argument is to attack the person rather than the 
argument that the person has made. It avoids a proper debate with 
a diversion rather than dealing with the substance of the argument 


This argument involves an appeal to popular belief to support an 
argument, with a claim that the majority of the population 
supports this argument. However, popular opinion is not always 
correct 


This argument is when an appeal is made to an authoritative figure 
to support an argument, and where the authority is not an expert in 
this area 

This is where the arguer tries to get people to accept a conclusion 
by making them feel sorry for someone 


The arguer makes the case that there is no conclusive evidence on 
the issue at hand, and therefore, his conclusion should be accepted 
as it has not been proven false 


The arguer sets up a version of an opponent’s position of his 
argument and defeats this watered down version of his opponent’s 
position 

This is a circular argument where the arguer relies on a premise 
that says the same thing as the conclusion and without providing 
any real evidence for the conclusion 


The arguer goes off on a tangent that has nothing to do with the 
argument in question. It is an irrelevant diversionary tactic 


The arguer presents the case that there are only two possible 
outcomes (often there are more that are excluded). One of the 
possible outcomes is then eliminated leading to the desired 
outcome. The argument suggests that there is only one outcome 


The Stoic school’ was founded in the Hellenistic period by Zeno of Citium (in 
Cyprus) in the late fourth/early third century B.C. The school presented its phi- 
losophy as a way of life, and it emphasized ethics as the main focus of human 


'The origin of the word Stoic is from the Stoa Poikile (S~tow TokKy), which was a covered 
walkway in the Agora of Athens. Zeno taught his philosophy in a public space at this location, and 
his followers became known as Stoics. 
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Fig. 14.1 Zeno of citium 


knowledge. The Stoics stressed the importance of living a good life in harmony 
with nature. (Fig. 14.1) 

The Stoics recognized the importance of reason and logic, and Chrysippus, the 
head of the Stoics in the third century B.C., developed an early version of 
propositional logic. This was a system of deduction in which the smallest unana- 
lyzed expressions are assertibles (Stoic equivalent of propositions). The assertibles 
have a truth-value such that at any moment of time they are either true or false. True 
assertibles are viewed as facts in the Stoic system of logic, and false assertibles are 
defined as the contradictories of true ones. 

Truth is temporal and assertions may change their truth-value over time. The 
assertibles may be simple or non-simple (more than one assertible), and there may 
be present tense, past tense and future tense assertibles. Chrysippus distinguished 
between simple and compound propositions, and he introduced a set of logical 
connectives for conjunction, disjunction and implication that are used to form 
non-simple assertibles from existing assertibles. 

The conjunction connective is of the form ‘both.. and..’, and it has two con- 
juncts. The disjunction connective is of the form ‘either.. or.. or..’, and it consists of 
two or more disjuncts. Conditionals are formed from the connective ‘if..,..’ and they 
consist of an antecedent and a consequence. 

His deductive system included various logical argument forms such as modus 
ponens and modus tollens. His propositional logic differed from syllogistic logic, in 
that the Stoic logic was based on propositions (or statements) as distinct from 
Aristotle’s term logic. However, he could express the universal affirmation in 
syllogistic logic (e.g. All As are B) by rephrasing it as a conditional statement that if 
something is A then it is B. 

Chrysippus’s propositional logic did not replace Aristotle’s syllogistic logic, and 
syllogistic logic remained in use up to the mid-nineteenth century. George Boole 
developed his symbolic logic in the mid-1800s, and his logic later formed the foun- 
dation for digital computing. Boole’s symbolic logic is discussed in the next section. 
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14.5 Boole’s Symbolic Logic 


George Boole (Fig. 14.2) was born in Lincoln, England, in 1815. His father (a 
cobbler who was interested in mathematics and optical instruments) taught him 
mathematics and showed him how to make optical instruments. Boole inherited his 
father’s interest in knowledge, and he was self-taught in mathematics and Greek. 
He taught at various schools near Lincoln, and he developed his mathematical 
knowledge by working his way through Newton’s Principia, as well as applying 
himself to the work of mathematicians such as Laplace and Lagrange. 

He published regular papers from his early twenties, and these included con- 
tributions to probability theory, differential equations and finite differences. He 
developed his symbolic algebra, which is the foundation for modern computing, 
and he is considered (along with Babbage) to be one of the grandfathers of com- 
puting. His work was theoretical, and he never actually built a computer or cal- 
culating machine. However, Boole’s symbolic logic was the perfect mathematical 
model for switching theory, and for the design of digital circuits. 

Boole became interested in formulating a calculus of reasoning, and he pub- 
lished a pamphlet titled “Mathematical Analysis of Logic” in 1847 [2]. This short 
book developed novel ideas on a logical method, and he argued that logic should be 
considered as a separate branch of mathematics, rather than a part of philosophy. He 
argued that there are mathematical laws to express the operation of reasoning in the 
human mind, and he showed how Aristotle’s syllogistic logic could be reduced to a 
set of algebraic equations. He corresponded regularly on logic with Augustus De 
Morgan.” 

He introduced two quantities ‘0’ and ‘1’ with the quantity 1 used to represent the 
universe of thinkable objects (i.e. the universal set), and the quantity 0 represents 
the absence of any objects (i.e. the empty set). He then employed symbols such as x, 
y, Z, etc. to represent collections or classes of objects given by the meaning attached 
to adjectives and nouns. Next, he introduced three operators (+, — and x ) that 
combined classes of objects. 

The expression x, y (i.e. x multiplied by y or x x y) combines the two classes x, 
y to form the new class x, y (i.e. the class whose objects satisfy the two meanings 
represented by the classes x and y). Similarly, the expression x + y combines the 
two classes x, y to form the new class x + y (that satisfies either the meaning 
represented by class x or class y). The expression xy combines the two classes x, 
y to form the new class x—y. This represents the class (that satisfies the meaning 
represented by class x but not class y. The expression (1 — x) represents objects that 
do not have the attribute that represents class x. 

Thus, if x= black and y=sheep, then xy represents the class of black 
sheep. Similarly, (1 — x) would represent the class obtained by the operation of 
selecting all things in the world except black things; x (1 — y) represents the class of 


De Morgan was a 19th British mathematician based at University College London. De Morgan’s 
laws in Set Theory and Logic state that: (AUB)° = A°MB¢ and ~(A VB) = ~A A -B. 
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Fig. 14.2 George Boole 


all things that are black but not sheep; and (1 — x) (1 — y) would give us all things 
that are neither sheep nor black. 

He showed that these symbols obeyed a rich collection of algebraic laws and 
could be added, multiplied, etc. in a manner that is similar to real numbers. These 
symbols may be used to reduce propositions to equations, and algebraic rules may 
be employed to solve the equations. The rules include: 


1. x+0=x (Additive Identity) 
2. x+(y+z) =(x+y)+z (Associative) 
3. xt+y=y+x (Commutative) 
4.x+(1-x)= 
5. xl =x (Multiplicative Identity) 
6. x0 = 
7.x+1= 
8. xy = yx (Commutative) 
9. x(yz) = (xy)z (Associative) 

10. x(y+z) =xyt+xz (Distributive) 

11. x(y =z) =xy— xz (Distributive) 

12. x? =x (idempotent) 


These operations are similar to the modern laws of set theory with the set union 
operation represented by ‘ + ’, and the set intersection operation is represented by 
multiplication. The universal set is represented by ‘1’ and the empty by ‘0’. The 
associative and distributive laws hold. Finally, the set complement operation is 
given by (1 — x). 

Boole applied the symbols to encode Aristotle’s Syllogistic Logic, and he 
showed how the syllogisms could be reduced to equations. This allowed conclu- 
sions to be derived from premises by eliminating the middle term in the syllogism. 
He refined his ideas on logic further in his book “An Investigation of the Laws of 
Thought” [3]. This book aimed to identify the fundamental laws underlying 


14.5  Boole’s Symbolic Logic 233 


reasoning in the human mind and to give expression to these laws in the symbolic 
language of calculus. 

He considered the equation x* = x to be a fundamental law of thought. It allows 
the principle of contradiction to be expressed (i.e. for an entity to possess an 
attribute and at the same time not to possess it) 


2 


Ko =X, 
Sx = 0, 
=> x(1—x) = 0. 


For example, if x represents the class of horses then (1 — x) represents the class of 
‘not-horses’. The product of two classes represents a class whose members are 
common to both classes. Hence, x (1 — x) represents the class whose members are at 
once both horses and ‘not-horses’, and the equation x (1 — x) = 0 expresses the fact 
that there is no such class. That is, it is the empty set. 

Boole contributed to other areas in mathematics including differential equations, 
finite differences’ and to the development of probability theory. Des McHale has 
written an interesting biography of Boole [4]. Boole’s logic appeared to have no 
practical use, but this changed with Claude Shannon’s 1937 Master’s Thesis, which 
showed its applicability to switching theory and to the design of digital circuits. 


14.5.1 Switching Circuits and Boolean Algebra 


Claude Shannon showed in his famous Master’s Thesis that Boole’s symbolic 
algebra provided the perfect mathematical model for switching theory and for the 
design of digital circuits. It may be employed to optimize the design of systems of 
electromechanical relays, and circuits with relays solve Boolean algebra problems. 
The use of the properties of electrical switches to process logic is the basic concept 
that underlies all modern electronic digital computers. Digital computers use binary 
digits O and 1, and Boolean logical operations may be implemented by electronic 
AND, OR and NOT gates. More complex circuits (e.g. arithmetic) may be designed 
from these fundamental building blocks. 

Modern electronic computers use billions of transistors that act as switches and 
can change states rapidly. The use of switches to represent binary values is the 
foundation of modern computing. A high voltage represents the binary value | with 
low voltage representing the binary value 0. 

A silicon chip may contain billions of tiny electronic switches arranged into 
logical gates. The basic logic gates are AND, OR and NOT, and these gates may be 
combined in various ways to allow the computer to perform more complex tasks 
such as binary arithmetic. Each gate has binary value inputs and outputs. 


Finite Differences are a numerical method used in solving differential equations. 
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Fig. 14.3 Binary AND 
operation 


Fig. 14.4 Binary OR 


operation —_|_> > 1 


The example in Fig. 14.3 is that of an “AND’ gate which produces the binary 
value | as output only if both inputs are 1. Otherwise, the result will be the binary 
value 0. Figure 14.4 is an ‘OR’ gate which produces the binary value | as output if 
any of its inputs is 1. Otherwise, it will produce the binary value 0. 

Finally, a NOT gate (Fig. 14.5) accepts only a single input which it reverses. 
That is, if the input is ‘1’ the value ‘0’ is produced and vice versa. 

The logic gates may be combined to form more complex circuits. The example 
in Fig. 14.6 is that of a half adder of 1 + 0. The inputs to the top OR gate are | and 
0 which yields the result of 1. The inputs to the bottom AND gate are 1 and 0 which 
yields the result 0, which is then inverted through the NOT gate to yield binary 1. 
Finally, the last AND gate receives two 1’s as input and the binary value | is the 
result of the addition. 

The half adder computes the addition of two arbitrary binary digits, but it does 


not calculate the carry. It may be extended to a full adder that provides a carry for 
addition. 
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Fig. 14.5 NOT operation 


Fig. 14.6 Half adder 


AND 


14.6 Application of Symbolic Logic to Digital Computing 


Claude Shannon (Fig. 14.7) was an American mathematician and engineer who 
made fundamental contributions to computing. He was the first person* to see the 
applicability of Boolean algebra to simplify the design of circuits and telephone 
routing switches. He showed that Boole’s symbolic logic developed in the nine- 
teenth century provided the perfect mathematical model for switching theory and 
for the subsequent design of digital circuits and computers. 

His influential Master’s Thesis is a key milestone in computing, and it shows 
how to layout circuits according to Boolean principles. It provides the theoretical 
foundation of switching circuits, and his insight of using the properties of electrical 
switches to do Boolean logic is the basic concept that underlies all electronic 
digital computers. 

Shannon realized that you could combine switches in circuits in such a manner 
as to carry out symbolic logic operations. This allowed binary arithmetic and more 
complex mathematical operations to be performed by relay circuits. He designed a 
circuit, which could add binary numbers, and he later designed circuits that could 
make comparisons, and thus is capable of performing a conditional statement. This 
was the birth of digital logic and the digital computing age. 

Vannevar Bush [5] was Shannon’s supervisor at MIT, and Shannon’s initial 
work was to improve Bush’s mechanical computing device known as the Differ- 
ential Analyser. This machine had a complicated control circuit that was composed 
of one hundred switches that could be automatically opened and closed by an 


“Victor Shestakov at Moscow State University also proposed a theory of electric switches based on 
Boolean algebra around the same time as Shannon. However, his results were published in Russian 
in 1941 whereas Shannon’s were published in 1937. 
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Fig. 14.7 Claude Shannon 


electromagnet. Shannon’s insight was his realization that an electronic circuit is 
similar to Boole’s symbolic algebra, and he showed how Boolean algebra could be 
employed to optimize the design of systems of electromechanical relays used in the 
analog computer. He also realized that circuits with relays could solve Boolean 
algebra problems. 

He showed in his Master’s thesis “A Symbolic Analysis of Relay and Switching 
Circuits” [6] that the binary digits (i.e. 0 and 1) can be represented by electrical 
switches. The implications of true and false being denoted by the binary digits one 
and zero were enormous since it allowed binary arithmetic and more complex 
mathematical operations to be performed by relay circuits. This provided elec- 
tronics engineers with the mathematical tool they needed to design digital electronic 
circuits and provided the foundation of digital electronic design. 

The design of circuits and telephone routing switches could be simplified with 
Boolean algebra. Shannon showed how to layout circuitry according to Boolean 
principles, and his Master’s thesis became the foundation for the practical design of 
digital circuits. These circuits are fundamental to the operation of modern com- 
puters and telecommunication systems, and his insight of using the properties of 
electrical switches to do Boolean logic is the basic concept that underlies all 
electronic digital computers. 


14.7 Frege 


Gottlob Frege (Fig. 14.8) was a German mathematician and logician who is con- 
sidered (along with Boole) to be one of the founders of modern logic. He also made 
important contributions to the foundations of mathematics, and he attempted to 
show that all of the basic truths of mathematics (or at least of arithmetic) could be 
derived from a limited set of logical axioms (this approach is known as logicism). 
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Fig. 14.8 Gottlob Frege 


He invented predicate logic and the universal and existential quantifiers, and 
predicate logic was a significant advance on Aristotle’s syllogistic logic. Predicate 
logic is described in more detail in Chap. 15. 

Frege’s first logical system, the 1879 Begriffsschrift, contained nine axioms and 
one rule of inference. It was the first axiomatization of logic, and it was complete in 
its treatment of propositional logic and first-order predicate logic. He published 
several important books on logic, including Begriffsschrift, in 1879; Die Grund- 
lagen der Arithmetik (The Foundations of Arithmetic) in 1884; and the two-volume 
work Grundgesetze der Arithmetik (Basic Laws of Arithmetic), which were pub- 
lished in 1893 and 1903. These books described his invention of axiomatic pred- 
icate logic; the use of quantified variables; and the application of his logic to the 
foundations of arithmetic. 

Frege presented his predicate logic in his books, and he began to use it to define 
natural numbers and their properties. He had intended producing three volumes of 
the Basic Laws of Arithmetic, with the later volumes dealing with the real numbers 
and their properties. However, Bertrand Russell discovered a contradiction in 
Frege’s system (see Russell’s paradox in Chap. 2), which he communicated to 
Frege shortly before the publication of the second volume. Frege was astounded by 
the contradiction and he struggled to find a satisfactory solution, and Russell later 
introduced the theory of types in the Principia Mathematica as a solution. 


14.8 Review Questions 


1. What is logic? 

2. What is a fallacy? 

3. Give examples of fallacies in arguments in natural language (e.g. in 
politics, marketing, debates) 
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4. Investigate some of the early paradoxes (for example the Tortoise and 
Achilles paradox or the arrow in flight paradox) and give your interpre- 
tation of the paradox. 

5. What is syllogistic logic and explain its relevance. 

6. What is stoic logic and explain its relevance. 

7. Explain the significance of the equation x° = x in Boole’s symbolic 
logic. 

8. Describe how Boole’s symbolic logic provided the foundation for 
digital computing. 

9. Describe Frege’s contributions to logic. 


14.9 Summary 


This chapter gave a short introduction to logic, which is concerned with reasoning 
and with establishing the validity of arguments. It allows conclusions to be deduced 
from premises according to logical rules, and the logical argument establishes the 
truth of the conclusion provided that the premises are true. 

The origins of logic are with the Greeks who were interested in the nature of 
truth. Socrates had a reputation for demolishing an opponent's position (it meant 
that he did not win any friends in debate), and the Socratean enquiry consisted of 
questions and answers in which the opponent would be led to a conclusion 
incompatible with his original position. His approach showed that his opponent’s 
position was incoherent and untenable. 

Aristotle did important work on logic, and he developed a system of logic, 
syllogistic logic, that remained in use up to the nineteenth century. Syllogistic logic 
is a ‘term logic’, with letters used to stand for the individual terms. A syllogism 
consists of two premises and a conclusion, where the conclusion is a valid deduction 
from the two premises. The Stoics developed an early form of propositional logic, 
where the assertibles (propositions) have a truth-value they are either true or false. 

George Boole developed his symbolic logic in the mid-1800s, and it later formed 
the foundation for digital computing. Boole argued that logic should be considered 
as a separate branch of mathematics, rather than a part of philosophy. He argued 
that there are mathematical laws to express the operation of reasoning in the human 
mind, and he showed how Aristotle’s syllogistic logic could be reduced to a set of 
algebraic equations. 

Gottlob Frege made important contributions to logic and to the foundations of 
mathematics. He attempted to show that all of the basic truths of mathematics (or at 
least of arithmetic) could be derived from a limited set of logical axioms (this 
approach is known as logicism). He invented predicate logic and the universal and 
existential quantifiers, and predicate logic was a significant advance on Aristotle’s 
syllogistic logic. 
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15.1. Introduction 


Logic is the study of reasoning and the validity of arguments, and it is concerned 
with the truth of statements (propositions) and the nature of truth. Formal logic is 
concerned with the form of arguments and the principles of valid inference. Valid 
arguments are truth preserving, and for a valid deductive argument, the conclusion 
will always be true if the premises are true. 

Propositional logic is the study of propositions, where a proposition is a state- 
ment that is either true or false. Propositions may be combined with other propo- 
sitions (with a logical connective) to form compound propositions. Truth tables are 
used to give operational definitions of the most important logical connectives, and 
they provide a mechanism to determine the truth-values of more complicated 
logical expressions. 

Propositonal logic may be used to encode simple arguments that are expressed in 
natural language, and to determine their validity. The validity of an argument may 
be determined from truth tables, or using the inference rules such as modus ponens 
to establish the conclusion via deductive steps. 

Predicate logic allows complex facts about the world to be represented, and new 
facts may be determined via deductive reasoning. Predicate calculus includes 
predicates, variables and quantifiers, and a predicate is a characteristic or property 
that the subject of a statement can have. A predicate may include variables, and 
statements with variables become propositions once the variables are assigned 
values. 

The universal quantifier is used to express a statement such as that all members 
of the domain of discourse have property P. This is written as (Vx)P(x), and it 
expresses the statement that the property P(x) is true for all x. 

The existential quantifier states that there is at least one member of the domain of 
discourse that has property P. This is written as (4x) P(x). 
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15.2 Propositional Logic 


Propositional logic is the study of propositions where a proposition is a statement 
that is either true or false. There are many examples of propositions such as 
“1 + 1 =2’ which is a true proposition, and the statement that “Today is Wed- 
nesday” which is true if today is Wednesday and false otherwise. The statement 
x > 0 is not a proposition as it contains a variable x, and it is only meaningful to 
consider its truth or falsity only when a value is assigned to x. Once the variable x is 
assigned a value, it becomes a proposition. The statement “This sentence is false” is 
not a proposition as it contains a self-reference that contradicts itself. Clearly, if the 
statement is true, it is false and if it is false, it is true. 

A propositional variable may be used to stand for a proposition (e.g. let the 
variable P stand for the proposition ‘2 + 2 = 4’ which is a true proposition). 
A propositional variable takes the value true or false. The negation of a proposition 
P (denoted —P) is the proposition that is true if and only if P is false, and is false if 
and only if P is true. 

A well-formed formula (wff) in propositional logic is a syntactically correct 
formula created according to the syntactic rules of the underlying calculus. 
A well-formed formula is built up from variables, constants, terms and logical 
connectives such as conjunction (and), disjunction (or), implication (if.. then..), 
equivalence (if and only if) and negation. A distinguished subset of these 
well-formed formulae are the axioms of the calculus, and there are rules of infer- 
ence that allow the truth of new formulae to be derived from the axioms and from 
formulae that have already been demonstrated to be true in the calculus. 

A formula in propositional calculus may contain several propositional variables, 
and the truth or falsity of the individual variables needs to be known prior to 
determine the truth or falsity of the logical formula. 

Each propositional variable has two possible values and a formula with n- 
propositional variables has 2” values associated with the n-propositional variables. 
The set of values associated with the n variables may be used to derive a truth table 
with 2” rows and n + | columns. Each row gives each of the 2” truth-values that the 
n variables may take, and column n + | gives the result of the logical expression for 
that set of values of the propositional variables. For example, the propositional 
formula W defined in the truth table above has two propositional variables A and B, 
with 2” = 4 rows for each of the values that the two propositional variables may 
take. There are 2 + 1 = 3 columns with W defined in the third column (Table 15.1). 

A rich set of connectives is employed in the calculus to combine propositions 
and to build up well-formed formulae. This includes the conjunction of two 
propositions (A A B); the disjunction of two propositions (A V B); and the impli- 
cation of two propositions (A — B). These connectives allow compound proposi- 
tions to be formed, and the truth of the compound propositions is determined from 
the truth-values of its constituent propositions and the rules associated with the 
logical connective. The meaning of the logical connectives is given by truth tables.! 


‘Basic truth tables were first used by Frege, and developed further by Post and Wittgenstein. 
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Table 15.1 Truth table for 


A B W (A, B) 
formula W T T T 

L F F 

F T F 

F F T 


Mathematical Logic is concerned with inference, and it involves proceeding in a 
methodical way from the axioms and using the rules of inference to derive further 
truths. 

The rules of inference allow new propositions to be deduced from a set of 
existing propositions. A valid argument (or deduction) is truth preserving, i.e. for a 
valid logical argument, if the set of premises is true, then the conclusion (i.e. the 
deduced proposition) will also be true. The rules of inference include rules such as 
modus ponens, and this rule states that given the truths of the proposition A, and the 
proposition A — B, then the truth of proposition B may be deduced. 

The propositional calculus is employed in reasoning about propositions, and it 
may be applied to formalize arguments in natural language. Boolean algebra is 
used in computer science, and it is named after George Boole, who was the first 
professor of mathematics at Queens College, Cork.* His symbolic logic (discussed 
in Chap. 14) is the foundation for modern computing. 


15.2.1. Truth Tables 


Truth tables give operational definitions of the most important logical connectives, 
and they provide a mechanism to determine the truth-values of more complicated 
compound expressions. Compound expressions are formed from propositions and 
connectives, and the truth-values of a compound expression containing several 
propositional variables are determined from the underlying propositional variables 
and the logical connectives. 

The conjunction of A and B (denoted A / B) is true if and only if both A and B are 
true, and is false in all other cases (Table 15.2). The disjunction of two propositions 
A and B (denoted A V B) is true if at least one of A and B are true, and false in all 
other cases (Table 15.3). The disjunction operator is known as the ‘inclusive or’ 
operator as it is also true when both A and B are true; there is also an exclusive or 
operator that is true exactly when one of A or B is true and is false otherwise. 


Example 15.1 Consider proposition A given by “An orange is a fruit” and 
proposition B given by ‘2 + 2 = 5’, then A is true and B is false. Therefore 


G) AAB (i.e. An orange is a fruit and 2 + 2 = 5) is false, 
Gi) AV B (i.e. An orange is a fruit or 2 + 2 = 5) is true. 


?This institution is now known as University College Cork and has over 20,000 students. 
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Table 15.2 Conjunction A B AAB 

T T T 

T F F 

F tT F 

F F F 
Table 15.3 Disjunction I B AVB 

T wT T 

T F T 

F w T 

F F F 
Table 15.4 Implication A B ANS iB 

T T T 

T F F 

F T T 

F F ay 


The implication operation (A — B) is true if whenever A is true; means that B is 
also true, and also whenever A is false (Table 15.4). It is equivalent (as shown by a 
truth table) to =A V B. The equivalence operation (A < B) is true whenever both 
A and B are true, or whenever both A and B are false (Table 15.5). 

The not operator (—) is a unary operator (i.e. it has one argument) and is such 
that A is true when A is false, and is false when A is true (Table 15.6). 


Table 15.5 Equivalence 


A B AB 

T T 1 

T F F 

F T F 

F F T 
Table 15.6 Not operation A a 

T F 
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Table 15.7 Truth table for 


T T F F 
T F T F 
sli F F T 
F T T als 
F T F F 
F F T F 
F F F F 


Example 15.2 Consider proposition A given by “Jaffa cakes are biscuits” and 
proposition B given by ‘2 + 2 = 5’, then A is true and B is false. Therefore 


Gi) A—B (e., Jaffa cakes are biscuits implies 2 + 2 = 5) is false, 
(ii) A-+<B (e., Jaffa cakes are biscuits is equivalent to 2 + 2 = 5) is false, 
(iti) —=B (ie., 2+ 2 #5) is true. 


Creating a Truth Table 


The truth table for a well-formed formula W(P,, Po, ..., P,) is a table with 2” rows 
and n + | columns. Each row lists a different combination of truth-values of the 
propositions P;, P, ..., P,, followed by the corresponding truth-value of W. 

The example above (Table 15.7) gives the truth table for a formula W with three 
propositional variables (meaning that there are 2° = 8 rows in the truth table). 


15.2.2 Properties of Propositional Calculus 


There are many well-known properties of propositional calculus such as the 
commutative, associative and distributive properties. These ease the evaluation of 
complex expressions and allow logical expressions to be simplified. 

The commutative property holds for the conjunction and disjunction operators, 
and it states that the order of evaluation of the two propositions may be reversed 
without affecting the resulting truth-value, i.e. 


AAB=BAA, 
AVB=BVA. 
The associative property holds for the conjunction and disjunction operators. 


This means that the order of evaluation of a sub-expression does not affect the 
resulting truth-value, i.e. 
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(AAB)\C=AA(BAOQ), 
(AVB) VC=AV(BVC). 


The conjunction operator distributes over the disjunction operator and vice 
versa. 


AA(BVC) =(AAB)V (AAO), 
AV (BAC) =(AVB)A(AVC). 


The result of the logical conjunction of two propositions is false if one of the 
propositions is false (irrespective of the value of the other proposition). 


AA\F=FAA=F. 


The result of the logical disjunction of two propositions is true if one of the 
propositions is true (irrespective of the value of the other proposition). 


AVT=TVA=T. 


The result of the logical disjunction of two propositions, where one of the 
propositions is known to be false is given by the truth-value of the other propo- 
sition. That is, the Boolean value ‘F’ acts as the identity for the disjunction 
operation. 


AVF=A=FVA. 


The result of the logical conjunction of two propositions, where one of the 
propositions is known to be true, is given by the truth-value of the other proposi- 
tion. That is, the Boolean value “T’ acts as the identity for the conjunction 
operation. 


AAT=A=TAA. 


The A and V operators are idempotent. That is, when the arguments of the 
conjunction or disjunction operator are the same, for proposition A, the result is A. 
The idempotent property allows expressions to be simplified. 


AAA=A, 
AVA=A. 
The law of the excluded middle is a fundamental property of the propositional 


calculus. It states that proposition A is either true or false, i.e. there is no third 
logical value. 
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Table 15.8 Tautology B B B\V-B 
BYV-B 
. T F T 
F T T 
AV -7A. 


We mentioned earlier that A — B is logically equivalent to =A V B (same truth 
table), and clearly =A V B is the same as —A V ~7~B = ——B V —A which is logi- 
cally equivalent to ~B — —A. In other words, A — B is logically equivalent to 
—=B — —A and this is known as the contrapositive. 

De Morgan was a contemporary of Boole in the nineteenth century, and the 
following law is known as De Morgan’s law. 


(A AB) =-AV -B, 
(A V B) =-A/A-B. 


Certain well-formed formulae are true for all values of their constituent vari- 
ables. This can be seen from the truth table when the last column of the truth table 
consists entirely of true values. 

A proposition that is true for all values of its constituent propositional variables 
is known as a tautology. An example of a tautology is the proposition A V =A 
(Table 15.8). 

A proposition that is false for all values of its constituent propositional variables 
is known as a contradiction. An example of a contradiction is the proposition 
AA WA. 


15.2.3 Proof in Propositional Calculus 


Logic enables further truths to be derived from existing truths by rules of inference 
that are truth preserving. Propositional calculus is both complete and consistent. 
The completeness property means that all true propositions are deducible in the 
calculus, and the consistency property means that there is no formula A such that 
both A and —A are deducible in the calculus. 

An argument in propositional logic consists of a sequence of formulae that are 
the premises of the argument and a further formula that is the conclusion of the 
argument. One elementary way to see if the argument is valid is to produce a truth 
table to determine if the conclusion is true whenever all of the premises are true. 

Consider a set of premises P,, P>, ... P,, and conclusion Q. Then to determine if 
the argument is valid using a truth table involves adding a column in the truth table 
for each premise P,, P2, ... P,,, and then to identify the rows in the truth table for 
which these premises are all true. The truth-value of the conclusion Q is examined 
in each of these rows, and if Q is true for each case for which P), Po, ... P, are all 
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true, then the argument is valid. This is equivalent to P} A P2A...\P,— Qisa 
tautology. 

An alternate approach to proof with truth tables is to assume the negation of the 
desired conclusion (i.e. =Q) and to show that the premises and the negation of the 
conclusion result in a contradiction (ie. Pi) \P2A...AP,A7Q) is a 
contradiction. 

The use of truth tables becomes cumbersome when there are a large number of 
variables involved, as there are 2” truth table entries for n- propositional variables. 


Procedure for Proof by Truth Table 


(i) Consider argument P), Po, ..., P,, with conclusion Q. 

Gi) Draw truth table with column in truth table for each premise P, Po, ..., Py. 
(iii) Identify rows in truth table for when these premises are all true. 
(iv) Examine truth-value of Q for these rows. 

(v) If Qis true for each case that P,, Po, ... P,, are true then the argument is valid. 
(vi) Thatis P} \P2A...AP, — Q is a tautology. 


Example 15.3 (Truth Tables) Consider the argument adapted from [1] and 
determine if it is valid. 

If the pianist plays the concerto, then crowd will come if the prices are not too high. 
If the pianist plays the concerto, then the prices will not be too high. 

Therefore, if the pianist plays the concerto, then crowd will come. 


Solution We will adopt a common proof technique that involves showing that the 
negation of the conclusion is incompatible (inconsistent) with the premises, and 
from this, we deduce that the conclusion must be true. First, we encode the argu- 
ment in propositional logic 

Let P stand for “The pianist plays the concerto”; C stands for “Crowds will 
come’; and H stands for “Prices are too high”. Then the argument may be expressed 
in propositional logic as 


P— (4H 0), 
P—-—H, 
POC. 


Then we negate the conclusion P — C and check the consistency of P > 
(AH — C) A (P > 7H) A-7(P = C)* using a truth table (Table 15.9). 

It can be seen from the last column in the truth table that the negation of the 
conclusion is incompatible with the premises, and therefore, it cannot be the case 
that the premises are true and the conclusion false. Therefore, the conclusion must 
be true whenever the premises are true, and we conclude that the argument is valid. 
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Table 15.9 Proof of argument with a truth table 


IP Cc AH vel dai oC P— (-H-C) IP => Val PoE (P > C) BS 
T T T F T 1 If T EF F 
w li F T T T T T FE F 
T F 1 Ii T T F EF T F 
T F F T F 1 T 18 T F 
F T al F T T T I IE F 
F T F T T T T 1 EF F 
F F W F T T T T iF F 
F F F T F T T T FE F 


Logical Equivalence and Logical Implication 

The laws of mathematical reasoning are truth preserving, and are concerned with 
deriving further truths from existing truths. Logical reasoning is concerned with 
moving from one line in mathematical argument to another and involves deducing 
the truth of another statement Q from the truth of P. 

The statement Q may be in some sense be logically equivalent to P and this 
allows the truth of Q to be immediately deduced. In other cases, the truth of P is 
sufficiently strong to deduce the truth of Q; in other words, P logically implies 
Q. This leads naturally to a discussion of the concepts of logical equivalence (W, = 
W3) and logical implication ( 

Ww, | Wo). 


Logical Equivalence 


Two well-formed formulae W, and W, with the same propositional variables (P, Q, 
R ...) are logically equivalent (W; = W>2) if they are always simultaneously true or 
false for any given truth-values of the propositional variables. 

If two well-formed formulae are logically equivalent then it does not matter 
which of W, and W; is used, and W; «> W) is a tautology. In Table 15.10 above, we 
see that P A Q is logically equivalent to =(—P V —Q). 


Logical Implication 


For two well-formed formulae W,; and W, with the same propositional variables (P, 
Q, R ...), W, logically implies W2 (W, L W,), if any assignment to the proposi- 
tional variables which makes W, true also makes W true (Table 15.11). That is, 
W, — Wy? is a tautology. 


Table 15.10 Logical Pp Q PAQ iP 1Q PV AQ ( PV Q) 
equivalence of two WFFs 

sla lj F F F T 

we F F W T F 

Bey F T F T F 

le | Ae F W ili T F 
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Table 15.11 Logical 

implication of S. WFFs BE CEO eek) 
TTT 
TTF 
TFT 
TFF 
Je IE a0 
FTF 
FFT 
FFF 


< 
wn 


Coes Mm Wile ie oie ot | 
ToS Se si6 


Example 15.4 Show by truth tables that (P A Q) V (Q A 7R) L (QO V R). 


The formula (P A Q) V (QA 7R) is true on rows 1, 2 and 6 and the formula 
(QV R) is also true on these rows. Therefore, (P A Q) V (Q A ~R) L (OV R). 


15.2.4 Semantic Tableaux in Propositional Logic 


We showed in Example 15.3 how truth tables may be used to demonstrate the 
validity of a logical argument. However, the problem with truth tables is that they 
can get extremely large very quickly (as the size of the table is 2” where n is the 
number of propositional variables), and so in this section, we will consider an 
alternative approach known as semantic tableaux. 

The basic idea of semantic tableaux is to determine if it is possible for a con- 
clusion to be false when all of the premises are true. If this is not possible, then the 
conclusion must be true when the premises are true, and so the conclusion is 
semantically entailed by the premises. The method of semantic tableaux is a 
technique to expose inconsistencies in a set of logical formulae, by identifying 
conflicting logical expressions. 

We present a short summary of the rules of semantic tableaux in Table 15.12, 
and we then proceed to provide proof for Example 15.3 using semantic tableaux 
instead of a truth table. 

Whenever a logical expression A and its negation —A appear in a branch of the 
tableau, then an inconsistency has been identified in that branch, and the branch is 
said to be closed. If all of the branches of the semantic tableaux are closed, then the 
logical propositions from which the tableau was formed are mutually inconsistent, 
and cannot be true together. 

The method of proof is to negate the conclusion and to show that all branches in 
the semantic tableau are closed, and therefore, it is not possible for the premises of 
the argument to be true and for the conclusion to be false. Therefore, the argument 
is valid and the conclusion follows from the premises. 
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Table 15.12 
Rule Definition 
No 
1 AAB 
A 
B 
2) AvB 
A B 
3 A->B 
=A B 
4 AoB 
AaB —A ~-=B 
5 a7A 
A 
=A =B 
Tl (A V B) 
aA 
—=B 
8 (A — B) 
A 
—=B 
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Rules of semantic tableaux 


Description 


If A A B is true, then both A and B are true, and may be added 
to the branch containing A A B 


If A V B is true, then either A or B is true, and we add two new 
branches to the tableaux, one containing A and one containing 
B 


If A — B is true, then either =A or B is true, and we add two 
new branches to the tableaux, one containing =A and one 
containing B 

If A < B is true, then either A A B or =A (A —B is true, and we 
add two new branches, one containing A / B and one 
containing —A A —B 


If ——A is true, then A may be added to the branch containing 
a7AA 

If =(A A B) is true, then either =A or —B is true, and we add 
two new branches to the tableaux, one containing =A and one 
containing —B 

If =(A V B) is true, then both =A and —B are true, and may be 
added to the branch containing —(A V B) 


If =(A — B) is true, then both A and —B are true, and may be 
added to the branch containing =(A — B) 


Example 15.5 (Semantic Tableaux): Perform the proof for Example 15.3 using 
semantic tableaux. 


Solution We formalized the argument previously as 


P— (AH > C) 
P— 7H 
P—C 


We negate the conclusion to get =(P — C) and we show that all branches in the 
semantic tableau are closed, and therefore, it is not possible for the premises of the 
argument to be true and for the conclusion false. Therefore, the argument is valid, 
and the truth of the conclusion follows from the truth of the premises. 
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closed =P (=H > C) 
none / \ 
closed —AH C 


H closed 


We have shown that all branches in the semantic tableau are closed, and 
therefore, it is not possible for the premises of the argument to be true and for the 
conclusion false. Therefore, the argument is valid as required. 


15.2.5 Natural Deduction 


The German mathematician, Gerhard Gentzen (Fig. 15.1), developed a method for 
logical deduction known as ‘Natural Deduction’, and his formal approach to natural 
deduction aimed to be as close as possible to natural reasoning. Gentzen worked as 
an assistant to David Hilbert at the University of Gdttingen, and he died of mal- 
nutrition in Prague at the end of the Second World War. 


Fig. 15.1 Gerhard Gentzen 
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Table 15.13 Natural deduction rules 


Rule Definition Description 
AI UPd Piers 1p Given the truth of propositions P), P2, ... P,, then the truth of the 
P,AP3A...AP,, 


conjunction P; A P2 \.../AP, follows. This rule shows how 
conjunction can be introduced 


AE Pi aPoA...AP, Given the truth the conjunction P; \ P2 A... A P, then the truth 
ip of proposition P; follows. This rule shows how a conjunction 
where i € {1,...,n} ies 
can be eliminated 


VI Given the truth of propositions P; then the truth of the 
disjunction P; V P) V...V P,, follows. This rule shows how a 


disjunction can be introduced 


VE Pauv...vP,,, P:>E.... P.-E Given the truth of the disjunction Py VP; V...V P,,, and that 
E each disjunct implies E, then the truth of E follows. This rule 
shows how a disjunction can be eliminated 


— J] From P,P)... P, inferP This rule states that if we have a theorem that allows P to be 


P; 
P, vP,v...vP,, 


(PiAP2A...P,) =P inferred from the truth of premises Pj, P2, ... P,,, then we can 
deduce (P; \P2A...AP,) — P. This is known as the 
Deduction Theorem 
—>E PP; P; This rule is known as modus ponens. The consequence of an 
P; implication follows if the antecedent is true (or has been 
previously proved) 
=I P-=P,,P/-P, If proposition P; implies proposition P; and vice versa then they 
PoP; are equivalent (i.e. P; > Pj) 
= PjoP; If proposition P; is equivalent to proposition P; then proposition 
P,P, P,P, P; implies proposition P; and vice versa 
aI FromPinfer P,P, If the proposition P allows a contradiction to be derived, then —P 
: is deduced. This is an example of a proof by contradiction 
=E_ From-P Hless If the proposition —P allows a contradiction to be derived, then P 


is deduced. This is an example of a proof by contradiction 


Natural deduction includes rules for A, V, — introduction and elimination and 
also for reductio ab adsurdum. There are ten inference rules in the Natural 
Deduction system, and they include inference rules for each of the five logical 
operators A, V, —, — and « . There are two inference rules per operator (an 
introduction rule and an elimination rule) in Table 15.13. 

Natural deduction may be employed in logical reasoning and is described in 
detail in [1, 2]. 


15.2.6 Sketch of Formalization of Propositional Calculus 


Truth tables provide an informal approach to proof and the proof is provided in 
terms of the meanings of the propositions and logical connectives. The formal- 
ization of propositional logic includes the definition of an alphabet of symbols and 
well-formed formulae of the calculus, the axioms of the calculus and rules of 
inference for logical deduction. 
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The deduction of a new formulae Q is via a sequence of well-formed formulae 
P\, Po, ... P, (where P,, = Q) such that each P; is either an axiom, a hypothesis, or 
deducible from an earlier pair of formula P;, P,, (where P, is of the form P; = P;) 
and modus ponens. Modus ponens is a rule of inference that states that given 
propositions A, and A => B then proposition B may be deduced. The deduction of a 
formula Q from a set of hypothesis H is denoted by H L Q, and where Q is 
deducible from the axioms alone this is denoted by - Q. 

The deduction theorem of propositional logic states that if H U {P} L Q, then 
H | p — Q, and the converse of the theorem is also true, i.e. if H -| P — Q then 
HU{P} |- Q. Formalism (this approach was developed by the German mathe- 
matician, David Hilbert) allows reasoning about symbols according to rules, and to 
derive theorems from formulae irrespective of the meanings of the symbols and 
formulae. 

Propositional calculus is sound, i.e. any theorem derived using the Hilbert 
approach is true. Further, the calculus is also complete, and every tautology has a 
proof (i.e. is a theorem in the formal system). The propositional calculus is con- 
sistent: (i.e. it is not possible that both the well-formed formula A and —A are 
deducible in the calculus). 

Propositional calculus is decidable, i.e. there is an algorithm (truth table) to 
determine for any well-formed formula A whether A is a theorem of the formal 
system. The Hilbert style system is slightly cumbersome in conducting proof and is 
quite different from the normal use of logic in the mathematical deduction. 


15.2.7 Applications of Propositional Calculus 


Propositional calculus may be employed in reasoning with arguments in natural 
language. First, the premises and conclusion of the argument are identified and 
formalized into propositions. Propositional logic is then employed to determine if 
the conclusion is a valid deduction from the premises. 

Consider, for example, the following argument that aims to prove that Superman 
does not exist: 

“Tf Superman were able and willing to prevent evil, he would do so. If Superman 
were unable to prevent evil he would be impotent; if he were unwilling to prevent 
evil he would be malevolent; Superman does not prevent evil. If superman exists he 
is neither malevolent nor impotent; therefore Superman does not exist”. 

First, letters are employed to represent the propositions as follows: 


Superman is able to prevent evil, 
Superman is willing to prevent evil, 
Superman is impotent, 

Superman is malevolent, 

Superman prevents evil, 

Superman exists. 


a) 


15.2 Propositional Logic 
Then, the argument above is formalized in propositional logic as follows: 


Premises 
P, (aA\w)—p, 
Py (7a—i)A(=w> mM), 
P3 ?, 
Py e—mnaiAnm. 


Conclusion P, \ Pz \ P3 \ P4 => 7e 


Proof that Superman does not exist 


1 aN\w—p Premise 1 

2) (>a — i) \ (=w > m) Premise 2 

3 =p Premise 3 

4 e— (ni Anam) Premise 4 

5 =p > 7(aAw) 1, Contrapositive 

6 a(aAw) 3,5 Modus Ponens 

vy, aa V aw 6, De Morgan’s Law 

8 a(-i Anam) > 7e 4, Contrapositive 

9 ivVm— —7e 8, De Morgan’s Law 

10 (7a — i) 2, \ Elimination 

11 (aw = m) 2, A Elimination 

12 Sa Wi 10, A — B equivalent to ~A V B 
13 anaViVm 11, V Introduction 

14 a7aV (i Vm) 

15 aa = (iV m) 14, A — B equivalent to =A V B 
16 aw Vm 11, A — B equivalent to >A VB 
7 saw V (iV m) 

18 aw (iV m) 17, A — B equivalent to =A V B 
19 (iV m) 7,15, 18 VElimination 

20 me 9,19 Modus Ponens 


Second Proof 


1 —p P3 

2 a(a Aw) V p P,\(A—>B=-—AVB) 
3 =(a Aw) 12 AVB,-B FA 

4 na aw 3, De Morgan’s Law 

5 (a = i) P, (A-Elimination) 

6 -a—iVm 5,x— y Lxoyvz 
7 (=w — m) P> (A-Elimination) 


(continued) 
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(continued) 
? P3 
aw > iVm ny) Lx—oyvz 
9 (=a V aw) = (i V m) Se VOR Lxvy—z 
10 (i V m) 4,9 Modus Ponens 
11 e— -(i Vm) P4 (De Morgan’s Law) 
12 ne V 7(i V m) 11(A += B=-—AVB) 
13 =e 10,12 AVB,-B FA 


Therefore, the conclusion that Superman does not exist is a valid deduction from 
the given premises. 


15.2.8 Limitations of Propositional Calculus 


The propositional calculus deals with propositions only. It is incapable of dealing 
with the syllogism “All Greeks are mortal; Socrates is a Greek; therefore Socrates is 
mortal”. This would be expressed in propositional calculus as three propositions A, 
B, therefore C, where A stands for “All Greeks are mortal’’, B stands for “Socrates is 
a Greek” and C stands for “Socrates is mortal”. Propositional logic does not allow 
the conclusion that all Greeks are mortal to be derived from the two premises. 

Predicate calculus deals with these limitations by employing variables and terms 
and using universal and existential quantification to express that a particular 
property is true of all (or at least one) values of a variable. Predicate calculus is 
discussed in the next section. 


15.3 Predicate Calculus 


Predicate logic is a richer system than propositional logic, and it allows complex 
facts about the world to be represented. It allows new facts about the world to be 
derived in a way that guarantees that if the initial facts are true, then the conclusions 
are true. Predicate calculus includes predicates, variables, constants and quantifiers. 

A predicate is a characteristic or property that an object can have, and we are 
predicating some property of the object. For example, “Socrates is a Greek’ could 
be expressed as G(s), with capital letters standing for predicates and small letters 
standing for objects. A predicate may include variables, and a statement with a 
variable becomes a proposition once the variables are assigned values. For example, 
G(x) states that the variable x is a Greek, whereas G(s) is an assignment of values to 
x. The set of values that the variables may take is termed the universe of discourse, 
and the variables take values from this set. 

Predicate calculus employs quantifiers to express properties such as all members 
of the domain have a particular property, e.g. (Vx) P(x), or that there is at least one 
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member that has a particular property, e.g. (4x)P(x). These are referred to as the 
universal and existential quantifiers. 

The syllogism “All Greeks are mortal; Socrates is a Greek; therefore Socrates is 
mortal” may be easily expressed in predicate calculus by 


(Vx)(G(x) > M(a)) 


In this example, the predicate G(x) stands for x is a Greek and the predicate M 
(x) stands for x is mortal. The formula G(x) — M(x) states that if x is a Greek, then 
x is mortal, and the formula (Vx)(G(x) — M(x)) states for any x that if x is a Greek, 
then x is mortal. The formula G(s) states that Socrates is a Greek and the formula M 
(s) states that Socrates is mortal. 


Example 15.6 (Predicates) A predicate may have one or more variables. A pred- 
icate that has only one variable (i.e. a unary or 1-place predicate) is often related to 
sets; a predicate with two variables (a 2-place predicate) is a relation; and a pred- 
icate with n variables (a n-place predicate) is a n-ary relation. Propositions do not 
contain variables and so they are 0-place predicates. The following are examples of 
predicates: 


(i) The predicate Prime(x) states that x is a prime number (with the natural 
numbers being the universe of discourse). 
(ii) Lawyer(a) may stand for a is a lawyer. 
(iii) Mean(m,x,y) states that m is the mean of x and y: ie., m= “(x + y). 
(iv) LT(,6) states that x is less than 6. 
(v) GT(x, 1) states that x is greater than m (where is the constant 3.14159). 
(vi) GE(x,y) states that x is greater than or equal to y. 
(vii) EQ, y) states that x is equal to y. 
(viii) LE(,y) states that x is less than or equal to y. 
(ix) Real(x) states that x is a real number. 
(x) Father(x,y) states that x is the father of y. 
(xi) -7(4x)(Prime(x) A B(x, 32,36)) states that there is no prime number 
between 32 and 36. 


— 
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Universal and Existential Quantification 


The universal quantifier is used to express a statement that all members of the 
domain have property P. This is written as (Vx)P(x) and expresses the statement hat 
the property. 

P(x) is true for all x. Similarly, (Vx1,%2, ..., %n)P(%1,%2, ---, Xn) States that 
property P(x1,%, ..., x,) is true for all x1,x9, ..., x,. Clearly, the predicate (Vx)P(a, b) 
is identical to P(a,b) since it contains no variables, and the predicate (Vy € 
N)(x<,y) is true if x = 1 and false otherwise. 

The existential quantifier states that there is at least one member in the domain of 
discourse that has property P. This is written as (4x)P(x) and the predicate 
(Ax, x2, ---, Xn)P(x1,%2, .--,; Xn) states that there is at least one value of (x1,2, ..., 
X,) such that P(%1,%2, ..., X,) iS true. 


Example 15.7 (Quantifiers) 


(i) (Sp) (Prime(p) A p > 1,000, 000) is true 


It expresses the fact that there is at least one prime number greater than a million, 
which is true as there is an infinite number of primes. 


(ii) (Vx) (Ay)x<y is true 


This predicate expresses the fact that given any number x we can always find a 
larger number, e.g. take y = x+1. 


(iii) (Sy) (Vx)x<y is false 


This predicate expresses the statement that there is a natural number y such that all 
natural numbers are less than y. Clearly, this statement is false since there is no 
largest natural number, and so the predicate (Sy) (Vx)x<y is false. 


Comment 15.1 /t is important. 

to be careful with the order in which quantifiers are written, as the meaning of a 
statement may be completely changed by the simple transposition of two 
quantifiers. 


The well-formed formulae in the predicate calculus are built from terms and 
predicates, and the rules for building the formulae are described briefly in 
Sect. 15.3.1. Examples of well-formed formulae include: 
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hoe 


(Vx) 

(Ax)? = 

(Vx) (x > d Ax<10), 

(Vy)x? = y, 

(Vx) (Ay)Love(y, x) (everyone is loved by someone), 
(Ay) (Vx)Love(y, x) (someone loves everyone). 


The formula (Vx)(x > 2) states that every x is greater than the constant 2; 
(Ax)x* = 2 states that there is an x that is the square root of 2; (Vx) (Ay)x? = y 
states that for every x there is a y such that the square of x is y. 


15.3.1 Sketch of Formalization of Predicate Calculus 


The formalization of predicate calculus includes the definition of an alphabet of 
symbols (including constants and variables), the definition of function and predicate 
letters, logical connectives and quantifiers. This leads to the definitions of the terms 
and well-formed formulae of the calculus. 

The predicate calculus is built from an alphabet of constants, variables, function 
letters, predicate letters and logical connectives (including the logical connectives 
discussed in propositional logic, and universal and existential quantifiers). 

The definition of terms and well-formed formulae specify the syntax of the 
predicate calculus, and the set of well-formed formulae gives the language of the 
predicate calculus. The terms and well-formed formulae are built from the symbols, 
and these symbols are not given meaning in the formal definition of the syntax. 

The language defined by the calculus needs to be given an interpretation in order 
to give meaning to the terms and formulae of the calculus. The interpretation needs 
to define the domain of values of the constants and variables, and to provide 
meaning to the function letters, the predicate letters and the logical connectives. 

Terms are built from constants, variables and function letters. A constant or 
variable is a term, and if t), to, ..., tf, are terms, then f*(t), to, ..-, t%) is a term (where 
f* is a k-ary function letter). Examples of terms include: 


x where x is a variable and square is a 1-ary function letter. 
+ y where x7 + ce is shorthand for the function add(square(x), square(y)). 


where add is a 2-ary function letter and square is a l-ary function 

letter. 

The well-formed formulae are built from terms as follows. If Pk is a k-ary 
predicate letter, t),t2, ..., t; are terms, then PK(t, to, ..-, t%) is a well-formed for- 
mula. If A and B are well-formed formulae, then so are —A,A A B,AVB,A — 
B,A = B, (Vx)A and (Ax)A. 
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There is a set of axioms for predicate calculus and two rules of inference used for 
the deduction of new formulae from the existing axioms and previously deduced 
formulae. The deduction of a new formula Q is via a sequence of well-formed 
formulae P,, P2, ... P, (where P,, = Q) such that each P; is either an axiom, a 
hypothesis, or deducible from one or more of the earlier formulae in the sequence. 

The two rules of inference are modus ponens and generalization. Modus ponens 
is a rule of inference that states that given predicate formulae A, and A => B then the 
predicate formula B may be deduced. Generalization is a rule of inference that states 
that given predicate formula A, then the formula (Vx)A may be deduced where x is 
any variable. 

The deduction of a formula Q from a set of hypothesis H is denoted by H . Q, 
and where Q is deducible from the axioms alone, this is denoted by L Q. The 
deduction theorem states that if HU {P} - Q then H L P — Q? and the converse 
of the theorem is also true, i.e. if H . P — Q then HU {P} - Q. 

The approach allows reasoning about symbols according to rules, and to derive 
theorems from formulae irrespective of the meanings of the symbols and formulae. 
Predicate calculus is sound i.e. any theorem derived using the approach is true, and 
the calculus is also complete. Predicate calculus is semi-decidable (not fully 
decidable) in the sense that the set of valid formulae is enumerable, which means 
that it is possible (in theory) to search through all of the valid formulae. 


Scope of Quantifiers 


The scope of the quantifier (Vx) in the well-formed formula (Vx)A is A. Similarly, 
the scope of the quantifier (4x) in the well-formed formula (4x)B is B. The variable 
x that occurs within the scope of the quantifier is said to be a bound variable. If a 
variable is not within the scope of a quantifier, it is free. 


Example 15.8 (Scope of Quantifiers) 


(i) x is free in the well-formed formula Vy(x? + y > 5), 
(ii) x is bound in the well-formed formula Vx(x* > 2). 


A well-formed formula is closed if it has no free variables. The substitution of a 
term f for x in A can only take place only when no free variable in t will become 
bound by a quantifier in A through the substitution. Otherwise, the interpretation of 
A would be altered by the substitution. 

A term tf is free for x in A if no free occurrence of x occurs within the scope of a 
quantifier (Vy) or (Sy) where y is free in ¢. This means that the term t may be 
substituted for x without altering the interpretation of the well-formed formula A. 

For example, suppose A is Vy(x? + y? > 2) and the term ¢ is y, then ¢ is not free 
for x in A as the substitution of ¢ for x in A will cause the free variable y in ¢ to 


3This is stated more formally that if HU {P} - Q by a deduction containing no application of 
generalization to a variable that occurs free in P then H L P—-Q. 
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become bound by the quantifier Vy in A, thereby altering the meaning of the formula 
to Vy(y? +y? > 2). 


15.3.2 Interpretation and Valuation Functions 


An interpretation gives meaning to a formula and it consists of a domain of dis- 
course and a valuation function. If the formula is a sentence (i.e. does not contain 
any free variables), then the given interpretation of the formula is either true or 
false. If a formula has free variables, then the truth or falsity of the formula depends 
on the values given to the free variables. A formula with free variables essentially 
describes a relation say, R(X1,X,.... X,) such that R(x), X, .... X,) is true if (4, Xo, .... 
Xn) is in relation R. If the formula is true irrespective of the values given to the free 
variables, then the formula is true in the interpretation. 

A valuation (meaning) function gives meaning to the logical symbols and 
connectives. Thus, associated with each constant c is a constant cy in some universe 
of values £; with each function symbol f of arity k, we have a function symbol fy in 
= and fy : XZ‘ — &; and for each predicate symbol P of arity k a relation PsC=*. 
The valuation function, in effect, gives the semantics of the language of the pred- 
icate calculus L. 

The truth of a predicate P is then defined in terms of the meanings of the terms, 
the meanings of the functions, predicate symbols, and the normal meanings of the 
connectives. 

Mendelson [3] provides a technical definition of truth in terms of satisfaction 
(with respect to an interpretation M). Intuitively a formula F is satisfiable if it is true 
(in the intuitive sense) for some assignment of the free variables in the formula F. If 
a formula F is satisfied for every possible assignment to the free variables in F, then 
it is true (in the technical sense) for the interpretation M. An analogous definition is 
provided for false in the interpretation M. 

A formula is valid if it is true in every interpretation, however, as there may be 
an uncountable number of interpretations, it may not be possible to check this 
requirement in practice. M is said to be a model for a set of formulae if and only if 
every formula is true in M. 

There is a distinction between proof theoretic and model theoretic approaches in 
predicate calculus. Proof theoretic is essentially syntactic, and there is a list of 
axioms with rules of inference. The theorems of the calculus are logically derived 
(i.e. L A) and the logical truths are a result of the syntax or form of the formulae, 
rather than the meaning of the formulae. Model theoretical, in contrast, is essen- 
tially semantic. The truth derives from the meaning of the symbols and connectives, 
rather than the logical structure of the formulae. This is written as L MA. 

A calculus is sound if all of the logically valid theorems are true in the inter- 
pretation, i.e. proof theoretic = model theoretic. A calculus is complete if all the 
truths in an interpretation are provable in the calculus, i.e. model theoretic = proof 
theoretic. A calculus is consistent if there is no formula A such that LA and - 7A. 
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The predicate calculus is sound, complete and consistent. Predicate calculus is 
not decidable (it is semi-decidable), i.e. there is no algorithm to determine for any 
well-formed formula A whether A is a theorem of the formal system. The unde- 
cidability of the predicate calculus may be demonstrated by showing that if the 
predicate calculus is decidable, then the halting problem (of Turing machines) is 
solvable (see Chap. 13). 


15.3.3 Properties of Predicate Calculus 


The following are properties of the predicate calculus: 


Gi) (Vx)P(x) = (Vy)P(), 
Gi) (Vx)P(x) = >(Ar) 4 P(x), 
(iii) = (Ax) P(x) = 7(Vx) = P(x), 
(iv) (Ar)P(x) = (Ay) P(y), 
(v) (Wx) (Vy)P(x,y) = (Vy) (Wx) PQ, y), 
(vi) (Ar)(P(x) V O(x)) = (Ax) P(x) V (Ay) Q0), 
(vii) (Vx) (P(x) A O(x)) = (Vx) P(x) A (Vy) Q(y) 


15.3.4 Applications of Predicate Calculus 


The predicate calculus is may be employed to formally state the system require- 
ments of a proposed system. It may be used to conduct formal proof to verify the 
presence or absence of certain properties in a specification. 

It may also be employed to define piecewise defined functions such as f(x,y), 
where f(x,y) is defined by: 


f(%y)=2%-y where x <0 A y<0; 
f(x,y) =r +y¥ where x > 0A y<0O; 
f(y) =xt+y wherex >O Ay =0; 
f(x,y) =x-y wherex<O0 A y= 0; 
f(x,y) =x+y wherex <O Ay > 0; 
f(xy) =e +y? wherex > OAy> 0. 


The predicate calculus may be employed for program verification and to show 
that a code fragment satisfies its specification. The statement that a program frag- 
ment F is correct with respect to its precondition P and postcondition Q is written as 
P{F}Q. The objective of program verification is to show that if the precondition is 
true before execution of the code fragment, then this implies that the postcondition 
is true after execution of the code. 
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A program fragment a is partially correct for precondition P and postcondition 
Q if and only if whenever a is executed in any state in which P is satisfied and 
execution terminates, then the resulting state satisfies Q. Partial correctness is 
denoted by P{F}Q, and Hoare’s Axiomatic Semantics is based on partial correct- 
ness. It requires proof that the postcondition is satisfied if the program terminates. 

A program fragment a is totally correct for precondition P and postcondition Q, 
if and only if whenever a is executed in any state in which P is satisfied then the 
execution terminates and the resulting state satisfies Q. It is denoted by {P}F{Q}, 
and Dijkstra’s calculus of weakest preconditions is based on total correctness [2, 4]. 
It is required to prove that if the precondition is satisfied then the program termi- 
nates and the postcondition is satisfied. Program verification is discussed in more 
detail in Chap. 20 and in [5]. 


15.3.5 Semantic Tableaux in Predicate Calculus 


We discussed the use of semantic tableaux for determining the validity of argu- 
ments in propositional logic earlier in this chapter, and its approach is to negate the 
conclusion of an argument and to show that this results in inconsistency with the 
premises of the argument. 

The use of semantic tableaux is similar to predicate logic, except that there are 
some additional rules to consider. As before, if all branches of a semantic tableau 
are closed, then the premises and the negation of the conclusion are mutually 
inconsistent. From this, we deduce that the conclusion must be true. 

The rules of semantic tableaux for propositional logic were presented in 
Table 15.12, and the additional rules specific to predicate logic are detailed in 
Table 15.14. 


Table 15.14 Extra rules of semantic tableaux (for predicate calculus) 


Rule Definition Description 
No 
1 (Vx)A(x) Universal instantiation 
A(t) where f is a term 
2 (Ax)A(x) Rule of Existential instantiation. The 


a) 
t 


A(t) where f is a term that has not been term is often a constant “a” 


used in the derivation so far 


3) 7(Vx)A(x) 
(Ax) sA(x) 
4 a(Ax)A(x) 


(Wx) A(x) 
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Example 15.9 (Semantic Tableaux) Show that the syllogism “All Greeks are 
mortal; Socrates is a Greek; therefore Socrates is mortal” is a valid argument in 
predicate calculus. 


Solution We expressed this argument previously as 


(Vx)(G(x) — M(x)); G(s); M(s). Therefore, we negate the conclusion (i.e. —M(s)). 
and try to construct a closed tableau. 


M(s) Universal Instantiation 


closed closed 


Therefore, as the tableau is closed, we deduce that the negation of the conclusion 


is inconsistent with the premises, and therefore, the conclusion follows from the 
premises. 


Example 15.10 (Semantic Tableaux) Determine whether the following argument is 
valid: 


All lecturers are motivated. 

Anyone who is motivated and clever will teach well. 
Joanne is a clever lecturer. 

Therefore, Joanne will teach well. 


Solution We encode the argument as follows: 


L(x) stands for ‘x is a lecturer’, 
M(x) stands for ‘x is motivated’, 
C(x) stands for ‘x is clever’, 

W(x) stands for ‘x will teach well’. 


We, therefore, wish to show that 


(Vx)(L(x) > M(x)) A (Vx)((M(x) A C(x)) — W(x)) A L(joanne) A C(joanne) 
E W(joanne) 
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Therefore, we negate the conclusion (i.e. ~W(joanne)) and try to construct a 


closed tableau: 


DANRWNE 


. (Vx)(L(x) > M(x)), 

. (Wx)((M(x) A C(x)) > W(x), 
. L(Gjoanne), 

. C(joanne), 

. 7Wjoanne) , 


L(joanne) + M(joanne) Universal Instantiation (line 1) 


(M(joanne) ~ C(joanne))— Wjoanne) Universal Instantiation (line 2) 


/\ 
—L(joanne) M(joanne) From line 6 


Closed 
/\ 


= (M(joanne) ~ C(joanne)) Wi(joanne) From line 7 


Closed 
/\ 


— M(joanne) = C(joanne) 


Therefore, since the tableau is closed, we deduce that the argument is valid. 


15.4 Review Questions 


ee 


. Draw a truth table to show that =(P — Q) = PA-Q. 

2. Translate the sentence “Execution of program P begun with x < 0 will not 
terminate” into propositional form. 

3. Prove the following theorems using the inference rules of natural 

deduction: 


(a) From b infer b V -c, 
(b) From b => (cA 4d),b infer d. 


4. Explain the difference between the universal and the existential quantifier. 
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5. Express the following statements in the predicate calculus: 


a. All natural numbers are greater than 10, 
b. There is at least one natural number between 5 and 10, 
c. There is a prime number between 100 and 200. 


6. Which of the following predicates are true? 


a. Wi € {10, ...,50}. i? <2000 \i<100 
b. FiEN. I >5A2=25 
c. SIEN. 2 = 25 


7. Use semantic tableaux to show that (A — A) V (B A -B) is true 
8. Determine if the following argument is valid. 


If Pilar lives in Cork, she lives in Ireland. Pilar lives in Cork. Therefore, Pilar 
lives in Ireland. 


15.5 Summary 


Propositional logic is the study of propositions, and a proposition is a statement that 
is either true or false. A formula in propositional calculus may contain several 
variables, and the truth or falsity of the individual variables, and the meanings of the 
logical connectives determine the truth or falsity of the logical formula. 

A rich set of connectives is employed in the calculus to combine propositions 
and to build up well-formed formulae. This includes the conjunction of two 
propositions (A A B), the disjunction of two propositions (A V B), and the impli- 
cation of two propositions (A = B). These connectives allow compound proposi- 
tions to be formed, and the truth of the compound propositions is determined from 
the truth-values of the constituent propositions and the rules associated with the 
logical connectives. 

Propositional calculus is both complete and consistent with all true propositions 
deducible in the calculus, and there is no formula A such that both A and 7A are 
deducible in the calculus. An argument in propositional logic consists of a sequence 
of formulae that are the premises of the argument and a further formula that is the 
conclusion of the argument. One elementary way to see if the argument is valid is to 
produce a truth table to determine if the conclusion is true whenever all of the 
premises are true. 

Predicate calculus allows expressions such as all members of the domain have a 
particular property to be expressed formally, e.g. (Vx)Px, or that there is at least one 


15.5 Summary 


member that has a particular property, e.g. (J 
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x)Px. Predicate calculus may be 


employed to specify the requirements for a proposed system and to give the defi- 
nition of a piecewise defined function. Semantic tableaux may be used for deter- 
mining the validity of arguments in propositional or predicate logic, and its 


approach is to negate the conclusion of an argum: 


ent and to show that this results in 


inconsistency with the premises of the argument. 
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16.1. Introduction 


In this chapter, we consider some advanced topics in logic including fuzzy logic, 
temporal logic, intuitionist logic, undefined values, logic and AI and theorem 
provers. Fuzzy logic is an extension of classical logic that acts as a mathematical 
model for vagueness, and it handles the concept of partial truth where truth-values 
lie between completely true and completely false. Temporal logic is concerned with 
the expression of properties that have time dependencies, and it allows temporal 
properties about the past, present and future to be expressed. 

Brouwer and others developed intuitionist logic as the logical foundation for 
intuitionism, which was a controversial theory of the foundations of mathematics 
based on a rejection of the law of the excluded middle and an insistence on con- 
structive existence. Martin L6f successfully applied it to type theory in the 1970s. 

Partial functions arise naturally in computer science, and such functions may fail 
to be defined for one or more values in their domain. One approach to deal with 
partial functions is to employ a precondition, which restricts the application of the 
function to values where it is defined. We consider three approaches to deal with 
undefined values, including the logic of partial functions, Dijkstra’s approach with 
his cand and cor operators and Parnas’s approach which preserves a classical 
two-valued logic. 

We examine the contribution of logic to the AI field and the work done by 
theorem provers to supporting proof. 


16.2 Fuzzy Logic 


Fuzzy logic is a branch of many-valued logic that allows inferences to be made 
when dealing with vagueness, and it can handle problems with imprecise or 
incomplete data. It differs from classical two-valued propositional logic, in that it is 
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based on degrees of truth, rather than on the standard binary truth-values of ‘true or 
false’ (1 or 0) of propositional logic. That is, while statements made in propositional 
logic are either true or false (1 or 0), the truth-value of a statement made in fuzzy 
logic is a value between 0 and 1. Its value expresses the extent to which the 
statement is true, with a value of | expressing absolute truth, and a value of 0 
expressing absolute falsity. 

Fuzzy logic uses degrees of truth as a mathematical model for vagueness, and 
this is useful since statements made in natural language are often vague and have a 
certain (rather than an absolute) degree of truth. It is an extension of classical logic 
to handle the concept of partial truth, where the truth-value lies between completely 
true and completely false. Lofti Zadeh developed fuzzy logic at Berkeley in the 
1960s, and it has been successfully applied to Expert Systems and other areas of 
Artificial Intelligence. 

For example, consider the statement “John is tall”. If John were 6 foot, 4 inches, 
then we would say that this is a true statement (with a truth value of 1) since John is 
well above average height. However, if John is 5 feet, 9 inches tall (around average 
height), then this statement has a degree of truth, and this could be indicated by a 
fuzzy truth valued of 0.6. Similarly, the statement that today is sunny may be 
assigned a truth-value of | if there are no clouds, 0.8 if there are a small number of 
clouds and 0 if it is raining all day. 

Propositions in fuzzy logic may be combined together to form compound 
propositions. Suppose X and Y are propositions in fuzzy logic, then compound 
propositions may be formed from the conjunction, disjunction and implication 
operators. The usual definition in fuzzy logic of the truth-values of the compound 
propositions formed from X and Y is given by 


Truth (=X) = 1—Truth (X), 
Truth (X and Y) = min(Truth(X), Truth(Y)), 
Truth (X or Y) = max(Truth(X), Truth(Y)), 
Truth (X — Y) = Truth(-X or Y). 


There is another way in which the operators may be defined in terms of 
multiplication: 


Truth (X and Y) = Truth(X) * Truth(Y), 
Truth (X or Y) = 1 — (1 — Truth(X)) « (1 — Truth(Y)). 


Truth (X — Y) = max {z| Truth(X) « z< Truth(Y)} where 0<z<1 
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Under these definitions, fuzzy logic is an extension of classical two-valued logic, 
which preserves the usual meaning of the logical connectives of propositional logic 
when the fuzzy values are just {0, 1}. 

Fuzzy logic has been useful in the expert system and artificial intelligence 
applications, and it has also been applied to the aerospace and automotive sectors, 
and to the medical, robotics and transport sectors. The first fuzzy logic controller 
was developed in England, in the mid-1970s. 


16.3. Temporal Logic 


Temporal logic is concerned with the expression of properties that have time 
dependencies, and the various temporal logics can express facts about the past, 
present and future. Temporal logic has been applied to specify temporal properties 
of natural language, artificial intelligence as well as the specification and verifica- 
tion of program and system behaviour. It provides a language to encode temporal 
knowledge in artificial intelligence applications, and it plays a useful role in the 
formal specification and verification of temporal properties (e.g. liveness and 
fairness) in safety- critical systems. 

The statements made in temporal logic can have a truth-value that varies over 
time. In other words, sometimes the statement is true and sometimes it is false, but it 
is never true or false at the same time. The two main types of temporal logics are 
linear time logics (reasoning about a single timeline) and branching time logics 
(reasoning about multiple timelines). 

The roots of temporal logic lie in work done by Aristotle in the fourth century B. 
C. when he considered whether a truth-value should be given to a statement about a 
future event that may or may not occur. For example, what truth value (if any) 
should be given to the statement that “There will be a sea battle tomorrow”. 
Aristotle argued against assigning a truth-value to such statements in the present 
time. 

Newtonian mechanics assumes an absolute concept of time independent of 
space, and this viewpoint remained dominant until the development of the theory of 
relativity in the early twentieth century (when space-time became the dominant 
paradigm). Arthur Prior began analyzing and formalizing the truth-values of 
statements concerning future events in the 1950s, and he introduced Tense Logic (a 
temporal logic) in the early 1960s. Tense logic contains four modal operators 
(strong and weak) that express events in the future or in the past: 


P (It has at some time been the case that), 
— F (it will be at some time be the case that), 
— H (it has always been the case that), 

G (It will always be the case that). 
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The P and F operators are known as weak tense operators, while the H and 
G operators are known as strong tense operators. The two pairs of operators are 
interdefinable via the equivalences: 


Pb © AHA 
Ho, =Pab 
Fo = G36 
Gob, & FA 


The set of formulae in Prior’s temporal logic may be defined recursively, and 
they include the connectives used in classical logic (e.g. —, A, V, —, <). We can 
express a property ¢ that is always true as Ad ~ Hd A > A Go, and a property that 
is sometimes true as Ed = Pb V b V Fo. Various extensions of Prior’s tense logic 
have been proposed to enhance its expressiveness, including the binary since 
temporal operator ‘S’, and the binary until temporal operator ‘U’. For example, the 
meaning of SW is that has been true since a time when y was true. 

Temporal logics are applicable in the specification of computer systems, and a 
specification may require safety, fairness and liveness properties to be expressed. 
For example, a fairness property may state that it will always be the case that a 
certain property will hold sometime in the future. The specification of temporal 
properties often involves the use of special temporal operators. 

We discuss common temporal operators that are used, including an operator to 
express properties that will always be true, properties that will eventually be true 
and a property that will be true in the next time instance. For example 


[]P  P is always true. 
¢P P will be true sometime in the future. 
OP P is true in the next time instant (discrete time). 


Linear Temporal Logic (LTL) was introduced by Pnueli in the late-1970s. This 
linear time logic is useful in expressing safety and liveness properties. Branching 
time logics assume a non-deterministic branching future for time (with a deter- 
ministic, linear past). Computation Tree Logic (CTL and CTL*) were introduced in 
the early 1980s by Emerson and others. 

It is also possible to express temporal operations directly in classical mathe- 
matics, and the well-known computer scientist, Parnas, prefers this approach. He is 
critical of computer scientists for introducing unnecessary formalisms when clas- 
sical mathematics already possesses the ability to do this. For example, the value of 
a function f at a time instance prior to the current time f¢ is defined as 


Prior(f,t) = lim,.of (t — 6). 
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For more detailed information on temporal logic, the reader is referred to the 
excellent article on temporal logic in [1]. 


16.4 Intuitionist Logic 


The controversial school of intuitionist mathematics was founded by the Dutch 
mathematician, L. E. J. Brouwer, who was a famous topologist, and well known for 
his fixpoint theorem in topology. This constructive approach to mathematics proved 
to be highly controversial, as its acceptance as a foundation of mathematics would 
have led to the rejection of many accepted theorems in classical mathematics (in- 
cluding his own fixed point theorem). 

Brouwer was deeply interested in the foundations of mathematics, and the 
problems arising from the paradoxes of set theory. He was determined to provide a 
secure foundation for mathematics, and his view was that an existence theorem in 
mathematics that demonstrates the proof of a mathematical object has no validity, 
unless the proof is constructive and accompanied by a procedure to construct the 
object. He, therefore, rejected indirect proof and the law of the excluded middle 
(P V =P) or equivalently (=P — P), and he insisted on an explicit construction of 
the mathematical object. 

The problem with the law of the excluded middle (LEM) arises in dealing with 
properties of infinite sets. For finite sets, one can decide if all elements of the set 
possess a certain property P by testing each one. However, this procedure is no 
longer possible for infinite sets. We may know that a certain element of the infinite 
set does not possess the property, or it may be the actual method of construction of 
the set that allows us to prove that every element has the property. However, the 
application of the law of the excluded middle is invalid for infinite sets, as we 
cannot conclude from the situation where not all elements of an infinite set possess 
a property P that there exists at least one element which does not have the property 
P. In other words, the law of the excluded middle may only be applied in cases 
where the conclusion can be reached in a finite number of steps. 

Consequently, if the Brouwer view of the world were accepted, then many of the 
classical theorems of mathematics (including his own well-known results in 
topology) could no longer be said to be true. His approach to the foundations of 
mathematics hardly made him popular with other mathematicians (the differences 
were so fundamental that it was more like a war), and intuitionism never became 
mainstream in mathematics. It led to deep and bitter divisions between Hilbert and 
Brouwer, with Hilbert accusing Brouwer (and Weyl) of trying to overthrow 
everything that did not suit them in mathematics, and that intuitionism was treason 
to science. Hilbert argued that a suitable foundation for mathematics should aim to 
preserve most of mathematics. Brouwer described Hilbert’s formalist program as a 
false theory that would produce nothing of mathematical value. For Brouwer, ‘to 
exist’ is synonymous with “constructive existence’, and constructive mathematics is 
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relevant to computer science, as a program may be viewed as the result obtained 
from a constructive proof of its specification. 

Brouwer developed one of the more unusual logics that have been invented 
(intuitionist logic), in which many of the results of classical mathematics were no 
longer true. Intuitionist logic may be considered the logical basis of constructive 
mathematics, and formal systems for intuitionist propositional and predicate logic 
were developed by Heyting and others [2]. 

Consider a hypothetical mathematical property P(x) of which there is no known 
proof (i.e. it is unknown whether P(x) is true or false for arbitrary x where x ranges 
over the natural numbers). Therefore, the statement Vx(P(x) V > P(x)) cannot be 
asserted with the present state of knowledge, as neither P(x) or P(x) has been 
proved. That is, unproved statements in intuitionist logic are not given an inter- 
mediate truth-value, and they remain of an unknown truth-value until they have 
been either proved or disproved. 

The intuitionist interpretation of the logical connectives is different from clas- 
sical propositional logic. A sentence of the form A V B asserts that either a proof of 
A or a proof of B has been constructed, and A V B is not equivalent to =(—A A -B). 
Similarly, a proof of A A B is a pair whose first component is a proof of A, and 
whose second component is a proof of B. The statement Vx-P(x) is not equivalent 
to 4x P(x) in intuitionist logic. 

Intuitionist logic was applied to Type Theory by Martin L6f in the 1970s [3]. 
Intuitionist type theory is based on an analogy between propositions and types, 
where A A B is identified with A x B, the Cartesian product of A and B. The ele- 
ments in the set A x B are of the form (a, b) where a€ A and b € B. The 
expression A V B is identified with A + B, the disjoint union of A and B. The 
elements in the set A + B are got from tagging elements from A and B, and they are 
of the form inl (a) for a € A and inr (b) for b € B. The left and right injections are 
denoted by inl and inr. 
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Total functions f : X — Y are functions that are defined for every element in their 
domain, and total functions are widely used in mathematics. However, there are 
functions that are undefined for one or more elements in their domain, and one 
example is the function y = 1/x. This function is undefined at x = 0. 

Partial functions arise naturally in computer science, and such functions may fail 
to be defined for one or more values in their domain. One approach to deal with 
partial functions is to employ a precondition, which restricts the application of the 
function to where it is defined. This makes it possible to define a new set (a proper 
subset of the domain of the function) for which the function is total over the new 
set. 
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Undefined terms often arise! and need to be dealt with. Consider, the example of 
the square root function yx taken from [4]. The domain of this function is the 
positive real numbers, and the following expression is undefined: 


(x > 0) A = Vx) V (#50) AQ = V—4)). 


The reason this is undefined is since the usual rules for evaluating such an 
expression involve evaluating each sub-expression and then performing the Boo- 
lean operations. However, when x < 0, the sub-expression y = /x is undefined, 
whereas when x > 0, the sub-expression y = /—x is undefined. Clearly, it is desir- 
able that such expressions be handled, and that for the example above, the 
expression would evaluate to be true. 

Classical two-valued logic does not handle this situation adequately, and there 
have been several proposals to deal with undefined values. Dijkstra’s approach is to 
use the cand and cor operators in which the value of the left-hand operand 
determines whether the right-hand operand expression is evaluated or not. Jone’s 
logic of partial functions [5] uses a three-valued logic* and Parnas’s® approach is an 
extension to the predicate calculus to deal with partial functions that preserve the 
two-valued logic. 


16.5.1 Logic of Partial Functions 


Jones [5] has proposed the logic of partial functions (LPFs) as an approach to deal 
with terms that may be undefined. This is a three-valued logic and a logical term 
may be true, false or undefined (denoted -L). The definition of the truth-functional 
operators used in classical two-valued logic is extended to three-valued logic. The 
truth tables for conjunction and disjunction are defined in Fig. 16.1. 

The conjunction of P and Q is true when both P and Q are true; false if one of 
P or Q is false; and undefined otherwise. The operation is commutative. The 
disjunction of P and Q(P V Q) is true if one of P or Q is true; false if both P and 
Q are false; and undefined otherwise. The implication operation (P — Q) is true 
when P is false or when Q is true; false when P is true and Q is false; and undefined 
otherwise. The equivalence operation (P < Q) is true when both P and Q are true 
or false; it is false when P is true and Q is false (and vice versa); and it is undefined 
otherwise (Fig. 16.2). 

The not operator (—) is a unary operator such —A is true when A is false, false 
when A is true and undefined when A is undefined (Fig. 16.3). 


'It is best to avoid undefinedness by taking care with the definitions of terms and expressions. 
The above expression would evaluate to true under Jones 3-valued logic of partial functions. 
The above expression evaluates to true for Parnas logic (a 2-valued logic). 
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Fig. 16.3 Negation 


The result of an operation may be known immediately after knowing the value of 
one of the operands (e.g. disjunction is true if P is true irrespective of the value of 
Q). The law of the excluded middle, i.e. A V =A does not hold in the three-valued 
logic, and Jones [5] argues that this is reasonable as one would not expect the 
following to be true: 


(/o=1) V C/o # 1). 
There are other well-known laws that fail to hold, such as: 
(i) EE, 


Gi) Deduction Theorem E, LE, does not justify LE, — E, unless it is known 
that E, is defined. 


Many of the tautologies of standard logic also fail to hold. 
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Parnas’s approach to logic is based on classical two-valued logic, and his philos- 
ophy is that truth-values should be true or false only,* and that there is no third 
logical value. It is an extension to predicate calculus to deal with partial functions. 
The evaluation of a logical expression yields the value ‘true’ or ‘false’ irrespective 
of the assignment of values to the variables in the expression. This allows the 
expression: (y = \/x)) V (y = / — x), that is undefined in classical logic to yield 
the value true. 

The advantages of his approach are that no new symbols are introduced into the 
logic and that the logical connectives retain their traditional meaning. This makes it 
easier for engineers and computer scientists to understand, as it is closer to their 
intuitive understanding of logic. 

The meaning of predicate expressions is given by first defining the meaning of 
the primitive expressions. These are then used as the building blocks for predicate 
expressions. The evaluation of a primitive expression RV) (where V is a 
comma-separated set of terms with some elements of V involving the application of 
partial functions) is false if the value of an argument of a function used in one of the 
terms of V is not in the domain of that function.” The following examples (Tables 
16.1 and 16.2) should make this clearer: 

These primitive expressions are used to build the predicate expressions, and the 
standard logical connectives are used to yield truth-values for the predicate 
expression. Parnas logic is defined in detail in [4]. 

The power of Parnas logic may be seen by considering a tabular expressions 
example [4]. The table below specifies the behaviour of a program that searches the 
array B for the value x. It describes the properties of the values of ‘7’ and ‘present’. 
There are two cases to be considered (Fig. 16.4): 


1. There is an element in the array with the value of x 
2. There is no such element in the array with the value of x. 


Clearly, from the example above the predicate expressions 4i, B[i] =x and 
-(Si, Bli] = x) are defined. One disadvantage of the Parnas approach is that some 
common relational operators (e.g. >, >, <, and <) are not primitive in the logic. 
However, these relational operators are then constructed from primitive operators. 
Further, the axiom of reflection does not hold in the logic. 


‘Tt seems strange to assign the value false to the primitive predicate calculus expression y = '/ 9. 
>The approach avoids the undefined logical value (&#xFOSE;) and preserves the two-valued logic. 
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Table 16.1 Examples of Parnas evaluation of undefinedness 


Expression x<0 x>0 

y=vx False True if y = Vx, False otherwise 
y='h False False 

y= x +x False True if y = x° + Vx, False otherwise 


Table 16.2 Example of undefinedness in array 


Expression He (il INN i ¢ {1..N} 
Bli] =x True if B[i] = x False 
Sih, Jil| = ae True if B[i] = x for some i, False otherwise False 
(i, Bii]=x) =(4 i, Bli]=x) Hb 
H Oe Bij '}=x true G 
present ’= true false 


Fig. 16.4 Finding index in array 


16.5.3 Dijkstra and Undefinedness 


The cand and cor operators were introduced by Dijkstra (Fig. 16.5) to deal with 
undefined values. They are non-commutative operators and allow the evaluation of 
predicates that contain undefined values. 

Consider the following expression: 


y=0V(/y = 2). 


Fig. 16.5 Edsger Dijkstra. 
Courtesy of Brian Randell 
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Table 16.3 a cand b 
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Then this expression is undefined when y = 0 as x/y is undefined since the 
logical disjunction operation is not defined when one of its operands is undefined. 
However, there is a case for giving meaning to such an expression when y = 0, 
since in that case, the first operand of the logical or operation is true. Further, the 
logical disjunction operation is defined to be true if either of its operands is true. 
This motivates the introduction of the cand and cor operators. These operators are 
associative and their truth tables are defined in Tables 16.3 and 16.4. 

The order of the evaluation of the operands for the cand operation is to evaluate 
the first operand; if the first operand is true then the result of the operation is the 
second operand; otherwise the result is false. The expression a cand b is equivalent to 


acand b & if athen belse F. 


The order of the evaluation of the operands for the cor operation is to evaluate 
the first operand. If the first operand is true, then the result of the operation is true; 
otherwise the result of the operation is the second operand. The expression a cor b 
is equivalent to 


acor b & if athen T else b. 


Table 16.4 acorb 
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The cand and cor operators satisfy the following laws: 
e Associativity 
The cand and cor operators are associative. 
(Acand B)cand C = A cand(B cand C), 


(Acor B) cor C = Acor(BeorC), 


e Distributivity 
The cand operator distributes over the cor operator and vice versa. 
Acand (B cor C) = (Acand B)cor(A cand C), 
Acor(B A C) = (Acor B)cand(A cor C). 
De Morgan’s law enables logical expressions to be simplified. 
—(A cand B) = —A cor-B, 


—=(A cor B) = =Acand —B. 


16.6 Logic and Al 


The long-term goal of Artificial Intelligence is to create a thinking machine that is 
intelligent, has consciousness, has the ability to learn, has free will and is ethical. 
Artificial Intelligence is a young field and John McCarthy and others coined the 
term in 1956. Alan Turing devised the Turing Test in the early 1950s as a way to 
determine whether a machine was conscious and intelligent. Turing believed that 
machines would eventually be developed that would stand a good chance of passing 
the “Turing Test’. 

There are deep philosophical problems in Artificial Intelligence, and some 
researchers believe that its goals are impossible or incoherent. Even if Artificial 
Intelligence is possible, there are moral issues to consider such as the exploitation of 
artificial machines by humans and whether it is ethical to do this. Weizenbaum 
argues that AI is a threat to human dignity, and that AI should not replace humans 
in positions that require respect and care. 
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Fig. 16.6 John McCarthy. 
Courtesy of John McCarthy 


4 


John McCarthy has long advocated the use of logic in AI, and mathematical 
logic has been used in the AI field to formalize knowledge and in guiding the 
design of mechanized reasoning systems. Logic has been used as an analytic tool, 
as a knowledge representation formalism and as a programming language 
(Fig. 16.6). 

McCarthy’s long-term goal was to formalize common-sense reasoning, i.e. the 
normal reasoning that is employed in problem-solving and dealing with normal 
events in the real world. McCarthy [6] argues that it is reasonable for logic to play a 
key role in the formalization of common-sense knowledge, and this includes the 
formalization of basic facts about actions and their effects, facts about beliefs and 
desires and facts about knowledge and how it is obtained. His approach allows 
common-sense problems to be solved by logical reasoning. 

Its formalization requires sufficient understanding of the common-sense world, 
and often the relevant facts to solve a particular problem are unknown. It may be 
that knowledge thought relevant may be irrelevant and vice versa. A computer may 
have millions of facts stored in its memory, and the problem is how to determine 
which of these should be chosen from its memory to serve as premises in logical 
deduction. 

McCarthy’s influential 1959 paper discusses various common-sense problems 
such as getting home from the airport. Mathematical logic is the standard approach 
to express premises, and it includes rules of inferences that are used to deduce valid 
conclusions from a set of premises. Its rigorous deductive reasoning shows how 
new formulae may be logically deduced from a set or premises. 

McCarthy’s approach to programs with common sense has been criticized by 
Bar-Hillel and others on the grounds that common sense is fairly elusive, and the 
difficulty that a machine would have in determining which facts are relevant to a 
particular deduction from its known set of facts. However, McCarthy’s approach 
has showed how logical techniques can contribute to the solution of specific AI 
problems. 
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Logic programming languages describe what is to be done, rather than how it 
should be done. These languages are concerned with the statement of the problem 
to be solved, rather than how the problem will be solved. These languages use 
mathematical logic as a tool in the statement of the problem definition. Logic is a 
useful tool in developing a body of knowledge (or theory), and it allows rigorous 
mathematical deduction to derive further truths from the existing set of truths. The 
theory is built up from a small set of axioms or postulates, and the rules of inference 
derive further truths logically. 

The objective of logic programming is to employ mathematical logic to assist 
with computer programming. Many problems are naturally expressed as a theory, 
and the statement of a problem to be solved is often equivalent to determining if a 
new hypothesis is consistent with an existing theory. Logic provides a rigorous way 
to determine this, as it includes a rigorous process for conducting proof. 

Computation in logic programming is essentially logical deduction, and logic 
programming languages use first-order® predicate calculus. They employ theorem 
proving to derive the desired truth from an initial set of axioms. These proofs are 
constructive’ in that more an actual object that satisfies the constraints is produced 
rather than a pure existence theorem. Logic programming specifies the objects, the 
relationships between them and the constraints that must be satisfied for the 
problem. 


— The set of objects involved in the computation, 
— The relationships that hold between the objects, 
— The constraints of the particular problem. 


The language interpreter decides how to satisfy the particular constraints. Arti- 
ficial Intelligence influenced the development of logic programming, and John 
McCarthy® demonstrated that mathematical logic could be used for expressing 
knowledge. The first logic programming language was Planner developed by Carl 
Hewitt at MIT in 1969. It uses a procedural approach for knowledge representation 
rather than McCarthy’s declarative approach. 

The best-known logic programming language is Prolog, which was developed in 
the early 1970s by Alain Colmerauer and Robert Kowalski. It stands for 
programming in logic. It is a goal-oriented language that is based on predicate 
logic. Prolog became an ISO standard in 1995. The language attempts to solve a 
goal by tackling the subgoals that the goal consists of: 


“First-order logic allows quantification over objects but not functions or relations. Higher-order 
logics allow quantification of functions and relations. 

"For example, the statement Sx such that x = /4 states that there is an x such that x is the square 
root of 4, and the constructive existence yields that the answer is that x =2 or x —2, ie. 
constructive existence provides more the truth of the statement of existence, and an actual object 
satisfying the existence criteria is explicitly produced. 

John McCarthy received the Turing Award in 1971 for his contributions to Artificial Intelligence. 
He also developed the programming language LISP. 
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goal :— subgoal,, ..., subgoal,,. 


That is, in order to prove a particular goal, it is sufficient to prove subgoal, 
through subgoal,,. Each line of a Prolog program consists of a rule or a fact, and the 
language specifies what exists rather than how. The following program fragment 
has one rule and two facts: 


Grandmother (G,S):- parent(P, S), mother(G, P). 
Mother (Sarah, Isaac). 
Parent (Isaac, Jacob). 


The first line in the program fragment is a rule that states that G is the grand- 
mother of S if there is a parent P of S and G is the mother of P. The next two 
statements are facts stating that Isaac is a parent of Jacob, and that Sarah is the 
mother of Isaac. A particular goal clause is true if all of its subclauses are true 


goalclause(V,): — clause; (Vj), .., clausem(Vm)- 

A Horn clause consists of a goal clause and a set of clauses that must be proven 
separately. Prolog finds solutions by unification, i.e. by binding a variable to a value. 
For an implication to succeed, all goal variables Vg on the left side of:- must find a 
solution by binding variables from the clauses which are activated on the right side. 
When all clauses are examined and all variables in Vg are bound, the goal succeeds. 
But if a variable cannot be bound for a given clause, then that clause fails. Following 
the failure, Prolog backtracks, and this involves going back to the left to previous 
clauses to continue trying to unify with alternative bindings. Backtracking gives 
Prolog the ability to find multiple solutions to a given query or goal. 

Logic programming languages generally use a simple searching strategy to 
consider the following alternatives: 


— If a goal succeeds and there are more goals to achieve, then remember any 
untried alternatives and go on to the next goal. 

— Ifa goal is achieved and there are no more goals to achieve, then stop with 
success. 

— Ifa goal fails and there are alternative ways to solve it, then try the next one. 

— Ifa goal fails and there are no alternate ways to solve it, and there is a previous 
goal, then go back to the previous goal. 

— Ifa goal fails and there are no alternate ways to solve it, and no previous goal, 
then stop with failure. 


Constraint programming is a programming paradigm where relations between 
variables can be stated in the form of constraints. Constraints specify the properties 
of the solution and differ from the imperative programming languages, in that they 
do not specify the sequence of steps to execute. For more detailed information on 
Artificial Intelligence; see Chap. 22 of [7]. 
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16.7 Review Questions 


e 


What is fuzzy logic? 

2. What is intuitionist logic and how is it different from classical logic? 

3. Discuss the problem of undefinedness and the advantages and disadvan- 
tages of three-valued logic. Describe the approaches of Parnas, Dijkstra 
and Jones. 

4. What is temporal logic? 

5. Show how the temporal operators may be expressed in classical mathe- 
matics. Discuss the merits of temporal operators. 

6. Investigate the Isabelle (or another) theorem-proving environment and 
determine the extent to which it may assist with proof. 

7. Discuss the applications of logic to AI. 


16.8 Summary 


We discussed some advanced topics in logic in this chapter, including fuzzy logic, 
temporal logic, intuitionist logic, undefined values, logic and AI and theorem 
provers. Fuzzy logic is an extension of classical logic that acts as a mathematical 
model for vagueness, whereas temporal logic is concerned with the expression of 
properties that have time dependencies. 

Intuitionism was a controversial school of mathematics that aimed to provide a 
solid foundation for mathematics. Its adherents rejected the law of the excluded 
middle and insisted that, for an entity to exist, there must be a constructive proof of 
its existence. Martin L6f applied intuitionistic logic to type theory in the 1970s. 

Partial functions arise naturally in computer science, and such functions may fail 
to be defined for one or more values in their domain. There are a number of 
approaches to deal with undefined values, including Jone’s logic of partial func- 
tions, Dijkstra’s cand and cor operators and Parnas’s approach which preserves a 
classical two-valued logic. 

We discussed temporal logic and its applications to the safety-critical field, 
including the specification of properties with time dependencies. We discussed the 
application of logic to the AI field, and logic has been used to formalize knowledge 
in AI systems. 
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17.1. Introduction 


The word ‘proof is generally interpreted as facts or evidence that support a par- 
ticular proposition or belief, and such proofs are generally conducted in natural 
language. Several premises (which are self-evident or already established) are 
presented, and from these premises (via deductive or inductive reasoning) further 
propositions are established until finally the conclusion is established. 

The proof of a theorem in mathematics requires additional rigour, and such 
proofs consist of a mixture of natural language and mathematical argument. It is 
common to skip over the trivial steps in the proof, and independent mathematicians 
conduct peer reviews to provide additional confidence in the correctness of the 
proof and to ensure that no unwarranted assumptions or errors in reasoning have 
been made. Proofs conducted in logic are extremely rigorous as every step in the 
proof is explicit.' 

Mathematical proof dates back to the Greeks, and many students are familiar 
with Euclid’s work (The Elements) in geometry, where, from a small set of axioms, 
postulates and definitions, he derived many of the well-known theorems of 
geometry. Euclid was a Hellenistic mathematician based in Alexandria around 300 
BC, and his style of proof was mainly constructive, i.e. in addition to the proof of 
the existence of an object, he actually constructed the object in the proof. Euclidean 
geometry remained unchallenged for over 2000 years, until the development of the 
non-Euclidean geometries in the nineteenth century, and these geometries were 
based on a rejection of Euclid’s controversial 5th postulate (the parallels postulate). 


'Perhaps a good analogy might be that a mathematical proof is like a program written in a 
high-level language such as C, whereas a formal mathematical proof in logic is like a program 
written in assembly language. 
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Mathematical proof may employ a “divide and conquer” technique, i.e. breaking 
the conjecture down into subgoals and then attempting to prove each of the sub- 
goals. Another common proof technique is indirect proof where we assume the 
opposite of what we wish to prove, and we show that this results in a contradiction 
(e.g. the proof that there are an infinite number of primes or the proof that there is 
no rational number whose square is 2 in Chap. 3). Other proof techniques used are 
the method of mathematical induction, which involves proof of the base case and 
inductive step (see Chap. 4). 

Aristotle developed syllogistic logic in the fourth century BC, and the rules of 
reasoning with valid syllogisms remained dominant in logic up to the nineteenth 
century. Boole develop his mathematical logic in the mid-nineteenth century, and 
he aimed to develop a calculus of reasoning to verify the correctness of arguments 
using logical connectives. Predicate logic (including universal and existential 
quantifiers) was introduced by Frege in the late nineteenth century as part of his 
efforts to derive mathematics from purely logical principles. Russell and Whitehead 
continued this attempt in Principia Mathematica, and Russell introduced the theory 
of types to deal with the paradoxes in set theory which he identified in Frege’s 
system. 

The formalists introduced extensive axioms in addition to logical principles, and 
Hilbert’s program led to the definition of a formal mathematical proof as a sequence 
of formulae, where each element is either an axiom or derived from a previous 
element in the series by applying a fixed set of mechanical rules (e.g. modus 
ponens). The last line in the proof is the theorem to be proved, and the formal proof 
is essentially syntactic following rules with the formulae simply a string of symbols 
and the meaning of the symbols is unimportant. 

The formalists later ran into problems in trying to prove that a formal system 
powerful enough to include arithmetic was both complete and consistent, and the 
results of Gédel showed that such a system would be incomplete (and one of the 
propositions without a proof is that of its own consistency). Turing later showed 
(with his Turing machine) that mathematics is undecidable, i.e. there is no algo- 
rithm or mechanical procedure that may be applied in a finite number of steps to 
determine if an arbitrary mathematical proposition is true or false. 

The proofs employed in mathematics are rarely formal (in the sense of Hilbert’s 
program), whereas they involve deductions from a set of axioms, these deductions 
are rarely expressed as the application of individual rules of logical inference. 

The application of formal methods in an industrial environment requires the use 
of machine-assisted proof since thousands of proof obligations arise from a formal 
specification, and theorem provers are essential in resolving these efficiently. Many 
proofs in formal methods are concerned with crosschecking the details of the 
specification, or checking the validity of the refinement steps, or checking that 
certain properties are satisfied by the specification. There are often many tedious 
lemmas to be proved, and theorem provers” are essential in dealing with these. 


Most existing theorem provers are difficult to use and are for specialist use only. There is a need to 
improve the usability of theorem provers. 
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Fig. 17.1 Idea of automated 
theorem proving Problem / 
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Machine proof is explicit and reliance on some brilliant insight is avoided. Proofs 
by hand in formal methods are notorious for containing errors or jumps in rea- 
soning, whereas machine proofs are explicit but are often extremely lengthy and 
essentially unreadable. 

Automated theorem proving (ATP) is difficult, as often mathematicians prove a 
theorem with an initial intuitive feeling that the theorem is true (Fig. 17.1). Human 
intervention to provide guidance or intuition improves the effectiveness of the 
theorem prover. There are several tools available to support theorem proving, and 
these include the Boyer—Moore theorem prover (known as NQTHM), the Isabelle 
theorem prover and the HOL system. 

The proof of various properties about a program increases confidence in its 
correctness. However, an absolute proof of correctness* is unlikely except for the 
most trivial of programs. A program may consist of legacy software that is assumed 
to work; a compiler that is assumed to work correctly creates it. Theorem provers 
are programs that are assumed to function correctly. The best that mathematical 
proof in formal methods can claim is increased confidence in the correctness of the 
software, rather than an absolute proof of correctness. 


17.2 Early Automation of Proof 


Early work on the automation of proof began in the 1950s with the beginning of 
work in the Artificial Intelligence field, where the early AI practitioners were trying 
to develop a ‘thinking machine’. One of the earliest programs developed was the 


3This position is controversial with others arguing that if correctness is defined mathematically 
then the mathematical definition (i.e., formal specification) is a theorem, and the task is to prove 
that the program satisfies the theorem. They argue that the proofs for non-trivial programs exist, 
and that the reason why there are not many examples of such proofs is due to a lack of 
mathematical specifications. 
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Logic Theorist (LT), which was presented at the Dartmouth conference on Artificial 
Intelligence in 1956 [1]. 

It was developed by Allen Newell and Herbert Simon, and it could prove 38 of 
the first 52 theorems from Russell and Whitehead’s Principia Mathematica. [2].* 
Russell and Whitehead had attempted to derive all mathematics from axioms and 
the inference rules of logic, and the LT program conducted proof from a small set of 
propositional axioms and deduction rules. Its approach was to start with the theo- 
rem to be proved, and to then search for relevant axioms and operators to prove the 
theorem. The Logic Theorist proved theorems in the propositional calculus, but it 
did not support predicate calculus. It used the five basic axioms of propositional 
logic and three rules of inference from the Principii to prove theorems.” 

LT demonstrated that computers had the ability to encode knowledge and 
information and to perform intelligent operations such as solving theorems in 
mathematics. The heuristic approach of the LT program tried to emulate human 
mathematicians, but could not guarantee that proof would be found for every valid 
theorem. 

If no immediate one-step proof could be found, then a set of subgoals was 
generated (these are formulae from which the theorem may be proved in one step) 
and proofs of these were then searched for, and so on. The program could use 
previously proved theorems in the course of developing a proof of a new theorem. 
Newell and Simon were hoping that the Logic Theorist would do more than just 
prove theorems in logic, and their goal was that it would attempt to prove theorems 
in a human-like way, especially with a selective search. 

However, in practice, the Logic Theorist search was not very selective in its 
approach, and the subproblems were considered in the order in which they were 
generated, and so there was no actual heuristic procedure (as in human problem 
solving) to guess at which subproblem was most likely to yield an actual proof. 
This meant that the Logic Theorist could, in practice, find only very short proofs, 
since as the number of steps in the proof increased, the amount of search required to 
find the proof exploded. 

The Geometry Machine was developed by Herbert Gelernter at the IBM 
Research Center in New York, in the late 1950s, with the goal of developing 
intelligent behaviour in machines. It differed from the Logic Theorist in that it 
selected only the valid subgoals (i.e. it ignored the invalid ones), and attempted to 
find proof of these. The Geometry Machine was successful in finding the solution to 
a large number of geometry problems taken from high-school textbooks in plane 
geometry. 


Russell is said to have remarked that he was delighted to see that the Principia Mathematica could 
be done by machine, and that if he and Whitehead had known this in advance that they would not 
have wasted 10 years doing this work by hand in the early twentieth century. 

Another possibility (though an inefficient and poor simulation of human intelligence) would be to 
start with the five axioms of the Principia, and to apply the three rules of inference to logically 
derive all possible sequences of valid deductions. This is known as the British Museum algorithm 
(as sensible as putting monkeys in front of typewriters to reproduce all of the books of the British 
Museum). 
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The logicians Hao Wang and Evert Beth (the inventor of semantic tableaux 
which was discussed in Chap. 15) were critical of the approaches of the AI pioneers 
and believed that mathematical logic could do a lot more. Wang and others 
developed a theorem prover for first-order predicate calculus in 1960, but it had 
serious limitations due to the combinatorial explosion. 

Alan Robinson’s work on theorem provers in the early 1960s led to a proof 
procedure termed ‘resolution’, which appeared to provide a breakthrough in the 
automation of predicate calculus theorem provers. A resolution theorem prover is 
essentially provided with the axioms of the field of mathematics in question, and the 
negation of the conjecture whose proof is sought. It then proceeds until a contra- 
diction is reached, where there is no possible way for the axioms to be true and for 
the conjecture to be false. 

The initial success of resolution led to excitement in the AI field where pioneers 
such as John McCarthy (see Chap. 16) believed that human knowledge could be 
expressed in predicate calculus,° and therefore, if the resolution was indeed suc- 
cessful for efficient automated theorem provers, then the general problem of Arti- 
ficial Intelligence was well on the way to a solution. However, while resolution led 
to improvements with the state explosion problem, it did not eliminate the problem. 

This led to a fall-off in research into resolution-based approaches to theorem 
proving, and other heuristic-based techniques were investigated by Bledsoe in the 
late 1970s. The field of logic programming began in the early 1970s with the 
development of the Prolog programming language (see Chap. 16). Prolog is in a 
sense an application of automated theorem proving, where problems are stated in 
the form of goals (or theorems) that the system tries to prove using a resolution 
theorem prover. The theorem prover generally does not need to be very powerful as 
many Prolog programs require only a very limited search, and a depth-first search 
from the goal backwards to the hypotheses is conducted. 

The Argonne Laboratory developed the Aura System in the early 1980s (it was 
later replaced by Otter), as an improved resolution-based automated theorem pro- 
ver, and this led to renewed interest in resolution-based approaches to theorem 
proving. There is a more detailed account of the nature of proof and theorem 
proving in [1]. 


17.3. Interactive Theorem Provers 


The challenges in developing efficient automated theorem provers led researchers to 
question whether an effective fully automated theorem prover was possible, and 
whether it made more sense to develop a theorem prover that could be guided by a 


°McCarthy’s viewpoint that predicate logic was the solution for the AI field was disputed by 
Minsksy and others (resulting in a civil war between the logicists and the proceduralists). The 
proceduralists argued that formal logic was an inadequate representation of knowledge for AI, and 
that predicate calculus was an overly rigid and inadequate framework. They argued that an 
alternative approach such as the procedural representation of knowledge was required. 
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human in its search for proof. This led to the concept of Interactive theorem proving 
(ITP) which involves developing formal proofs by man—machine collaboration and 
is (in a sense) a new way of doing mathematics in front of a computer. 

Such a system is potentially useful in mathematical research in formalizing and 
checking proofs, and it allows the user to concentrate on the creative parts of the 
proof and relieves the user from the need of carrying out the trivial steps in the 
proof. It is also a useful way of verifying the correctness of published mathematical 
proofs by acting as a proof checker, where the ITP is provided with a formal proof 
constructed by a human, which may then be checked for correctness.’ Such a 
system is important in program verification in showing that the program satisfies its 
specification, especially in the safety/security-critical field. 

A group at Princeton developed a series of systems called Semi-automated 
mathematics (SAM) in the late 1960s, which combined logic routines with human 
guidance and control. Their approach placed the mathematician at the heart of the 
theorem proving, and it was a departure from the existing theorem proving 
approaches where the computer attempted to find proofs unaided. SAM provided 
proof of an unproven conjecture in lattice theory (SAM’s lemma), and this is 
regarded as the first contribution of automated reasoning systems to mathematics [1]. 

De Bruijn and others at the Technische Hogeschool in Eindhoven in the 
Netherlands commenced development of the Automath system in the late 1960s. 
This was a large-scale project for the automated verification of mathematics, and it 
was tested by treating a full textbook. Automath systematically checked the proofs 
from Landau’s text Grundlagen der Analysis (this foundation of analysis text was 
first published in 1930). 

The typical components of an Interactive Theorem Prover include an interactive 
proof editor to allow editing of proofs, formulae and terms in a formal theory of 
mathematics, and a large library of results which is essential for achieving complex 
results. 

The Gypsy verification environment and its associated theorem prover was 
developed at the University of Texas in the 1980s, and it achieved early success in 
program verification with its verification of the encrypted packet interface program 
(a 4200 line program). It supports the development of software systems and formal 
mathematical proof of their behaviour. 

The Boyer-Moore Theorem prover (NQTHM) was developed in the 
1970s/1980s at the University of Texas by B. S. Boyer and J. S. Moore [3]. It has 
been improved since then and it is currently known as NQTHM (it has been 
superseded by ACL2 available from the University of Texas). It supports mathe- 
matical induction as a rule of inference, and induction is a useful technique in 
proving the properties of programs. The axioms of Peano arithmetic are built into 
the theorem prover, and new axioms added to the system need to pass a ‘cor- 
rectness test’ to prevent the introduction of inconsistencies. 


7A formal mathematical proof (of a normal proof) is difficult to write down and can be lengthy. 
Mathematicians were not really interested in these proof checkers. 
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It is far more automated than many other interactive theorem provers, but it 
requires detailed human guidance (with suggested lemmas) for difficult proofs. The 
user, therefore, needs to understand the proof being sought and the internals of the 
theorem prover. 

It has been effective in proving well-known theorems such as Goedel’s 
Incompleteness Theorem, the insolvability of the Halting problem, a formalization 
of the Motorola MC 68020 Microprocessor and many more. 

Computational Logic Inc. was a company founded by Boyer and Moore in 1983 
to share the benefits of a formal approach to software development with the wider 
computing community. It was based in Austin, Texas, and provided services in the 
mathematical modelling of hardware and software systems. This involved the use of 
mathematics and logic to formally specify microprocessors and other systems. The 
use of its theorem prover was to formally verify that the implementation meets its 
specification, i.e. to prove that the microprocessor or other system satisfies its 
specification. 

The HOL system was developed at Cambridge University in the UK, and it is an 
environment for interactive theorem proving in a higher-order logic. It has been 
applied to the formalization of mathematics and to the verification of hardware 
(including the verification of microprocessor design). It requires skilled human 
guidance and is one of the most widely used theorem provers. It was originally 
developed in the early 1980s, and HOL 4 is the latest version. It is an open-source 
project and is used by academia and industry. 

Isabelle is a theorem-proving environment developed at Cambridge University 
by Larry Paulson and Tobias Nipkow of the Technical University of Munich. It 
allows mathematical formulas to be expressed in a formal language and provides 
tools for proving those formulas. The main application is the formalization of 
mathematical proofs, and proving the correctness of computer hardware or software 
with respect to their specification, and proving properties of computer languages 
and protocols. 

Isabelle is a generic theorem prover in the sense that it has the capacity to accept a 
variety of formal calculi, whereas most other theorem provers are specific to a specific 
formal calculus. Isabelle is available free of charge under an open-source license. 

There is a steep learning curve with the theorem provers above and it generally 
takes a couple of months for users to become familiar with them. However, 
automated theorem proving has become a useful tool in the verification of inte- 
grated circuit design. Several semiconductor companies use automated theorem 
proving to demonstrate the correctness of division and other operators on their 
processors. We present a selection of theorem provers in the next section. 
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17.4 A Selection of Theorem Provers 


Table 17.1 presents a small selection of the available automated and interactive 


theorem provers. 


17.5 Review Questions 


ee ge EO [oS 


What is a mathematical proof? 

What is a formal mathematical proof? 

What approaches are used to prove a theorem? 

What is a Theorem Prover? 

What role can theorem provers play in software development? 


Table 17.1 Selection of theorem provers 


Theorem prover 
ACL2 


OTTER 


PVS 


Theorem Proving 
System (TPS) 


HOL and Isabelle 


Boyer—Moore 


Description 


A Computational Logic for Applicative Common Lisp (ACL2) is part of the 
Boyer—Moore family of theorem provers. It is a software system consisting 
of a programming language (LISP) and an interactive theorem prover. It was 
developed in the mid-1990s as an industrial strength successor to the Boyer— 
Moore Theorem prover (NQTHM). It is used in the verification of 
safety-critical hardware and software, and in industrial applications such as 
the verification of the floating point module of a microprocessor 


OTTER is a resolution-style theorem prover for first order logic developed at 
the Argonne Laboratory at the University of Chicago (it was the successor to 
Aura). It has been mainly applied to abstract algebra and formal logic 


The Prototype Verification System (PVS) is a mechanized environment for 
formal specification and verification. It includes a specification language 
integrated with support tools and an interactive theorem prover. It was 
developed by SRI in California by John Rushby and others. The 
specification language is based on higher-order logic, and the theorem 
prover is guided by the user in conducting proof. It has been applied to the 
verification of hardware and software 


TPS is an automated theorem prover for first-order and higher-order logic (it 
can also prove theorems interactively). It was developed at Carnegie Mellon 
University, and is used for hardware and software verification 


HOL and Isabelle were developed by the Automated Reasoning Group at 
the University of Cambridge. The HOL system is an environment for 
interactive theorem proving in a higher-order logic, and it has been applied 
to hardware verification. Isabelle is a generic proof assistant which allows 
mathematical formulae to be expressed in a formal language, and it provides 
tools for proving those formulae in a logical calculus 

The Boyer—Moore Theorem prover (NQTHM) was developed at the 
University of Texas in the 1970s with the goal of checking the correctness of 
computer systems. It has been used to verify the correctness of 
microprocessors, and it has been superseded by ACL2 
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6. What is the difference between an automated theorem prover and an 
interactive theorem prover? 

7. Investigate and give a detailed description of one of the theorem provers 
in Table 17.1 


17.6 Summary 


A mathematical proof includes natural language and mathematical symbols, and 
often many of the tedious details of the proof are omitted. The proofs in mathematics 
are rarely formal as such, and many proofs in formal methods are concerned with 
crosschecking the details of the specification, or checking the validity of the 
refinement steps, or checking that certain properties are satisfied by the specification. 

Machine proof is explicit and reliance on some brilliant insight is avoided. 
Proofs by hand often contain errors or jumps in reasoning, while machine proofs are 
often extremely lengthy and unreadable. The application of formal methods in an 
industrial environment requires the use of machine-assisted proof since thousands 
of proof obligations arise, and theorem provers are essential in resolving these 
efficiently. The proof of various properties about a program increases confidence in 
its correctness. However, an absolute proof of correctness is unlikely except for the 
most trivial of programs. 

Automated theorem proving is difficult, as often mathematicians prove a theorem 
with an initial intuitive feeling that the theorem is true. Human intervention to 
provide guidance or intuition improves the effectiveness of the theorem prover. 
Early work on the automation of proof began in the 1950s with the beginning of 
work in the Artificial Intelligence field, and one of the earliest programs developed 
was the Logic Theorist, which was presented at the Dartmouth conference on 
Artificial Intelligence in 1956. 

The challenges in developing effective automated theorem provers led 
researchers to investigate whether it made more sense to develop a theorem prover 
that could be guided by a human in its search for proof. This led to the development 
of Interactive theorem proving which involved developing formal proofs by man— 
machine collaboration. 

The typical components of an Interactive Theorem Prover include an interactive 
proof editor to allow editing of proofs, formulae and terms in a formal theory of 
mathematics, and a large library of results which is essential for achieving complex 
results. An interactive theorem prover allows the user to concentrate on the creative 
parts of the proof and relieves the user from the need to carry out and verify the 
trivial steps in the proof. It is also a useful way of verifying the correctness of 
published mathematical proofs by acting as a proof checker and is also useful in 
program verification in showing that the program satisfies its specification. 
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18.1. Introduction 


The NATO Science Committee organized two famous conferences on software 
engineering in the late 1960s. The first conference was held in Garmisch, Germany, 
in 1968, and it was followed by a second conference in Rome, in 1969. The 
Garmisch conference was attended by over fifty people from eleven countries. 

The conferences highlighted the problems that existed in the software sector in 
the late 1960s, and the term “software crisis” was coined to refer to these problems. 
These included budget and schedule overruns of projects and problems with the 
quality and reliability of the delivered software. This conference led to the birth of 
software engineering as a separate discipline and the realization that programming 
is quite distinct from science and mathematics. Programmers are like engineers in 
the sense that they design and build products. Therefore, they need an appropriate 
software engineering education (not just on the latest technologies but on the 
fundamentals of engineering) in order to properly design and develop software. 

The construction of bridges was problematic in the nineteenth century, and many 
people who presented themselves as qualified to design and construct bridges did 
not have the required knowledge and expertise. Consequently, many bridges col- 
lapsed, endangering the lives of the public. This led to legislation requiring an 
engineer to be licensed by the Professional Engineering Association prior to 
practicing as an engineer. These engineering associations identify a core body of 
knowledge that the engineer is required to possess, and the licensing body verifies 
that the engineer has the required qualifications and experience. The licensing of 
engineers by most branches of engineering ensures that only personnel competent 
to design and build products actually do so. This, in turn, leads to products that the 
public can safely use. In other words, the engineer has a responsibility to ensure that 
the products are properly built, and are safe for the public to use. 
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Parnas argues that traditional engineering is contrasted with the software engi- 
neering discipline where there is no licensing mechanism, and where individuals 
with no qualifications can participate in the design and building of software 
products.’ However, best practice in modern HR places a strong emphasis on the 
qualification of staff. 

The Standish group has conducted research since the late 1990s [1] on the extent 
of problems with schedule and budget overruns of IT projects. The results indicate 
serious problems with on-time delivery, cost overruns and quality.” Fred Brooks 
has argued that software is inherently complex and that there is no silver bullet that 
will resolve all of the problems associated with software projects such as schedule 
overruns and software quality problems [2, 3]. 

Poor quality software can at best cause minor irritation to clients, and in some 
circumstances, it may seriously disrupt the work of the client organization leading 
to injury or even the death of individuals (e.g. as in the case of the Therac-25 
radiotherapy machine in the mid-1980s*). The Y2K problem occurred due to poor 
design, as the representation of the date used two digits to record the year rather 
than four. Its correction required major rework, as it was necessary to examine all 
existing software code to determine how the date was represented and to make 
appropriate corrections. Clearly, well-designed programs would have hidden the 
representation of the date , thereby minimizing the changes required for the year 
2000 compliance. 

Mathematics plays a key role in engineering, and it may potentially assist 
software engineers in delivering high-quality software products that are safe to use. 
Several mathematical approaches that may assist in delivering high-quality software 
described in [4]. However, it is important to recognize that while the use of 
mathematics is suitable for some areas of software engineering (especially in the 
safety and security-critical fields), less rigorous techniques (such as software 
inspections and testing) are sufficient in most other areas of software engineering. 


‘Modern HR recruitment specifies the requirements for a particular role, and the interviews 
establish whether the candidate is suitably qualified, and has the appropriate experience for the 
role. Parnas is arguing against the content of courses that emphasize the latest technologies rather 
than the fundamentals of engineering. 

“It should be noted that these are IT projects covering diverse sectors including banking, 
telecommunications, etc. rather than pure software companies. Mature software companies using 
the CMM tend to be more consistent in project delivery with high quality. 

3Therac-25 was a radiotherapy machine produced by the Atomic Energy of Canada Limited 
(AECL). It was involved in at least six accidents between 1985 and 1987 in which patients were 
given massive overdoses of radiation. The dose given was over 100 times the intended dose and 
three of the patients died from radiation poisoning. These accidents highlighted the dangers of 
software control of safety-critical systems. The investigation subsequently highlighted the poor 
software design of the system and the poor software development practices employed. 
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There is a lot of industrial interest in approaches to mature software engineering 
practices in software organizations (e.g. the use of software process maturity 
models such as the CMMI). These include approaches to assess and mature the 
software engineering processes in software companies, and they are described in 
[5, 6].* Software process improvement focuses mainly on improving the effec- 
tiveness of the management, engineering and organization practices related to 
software engineering. 


18.2 What is Software Engineering? 


Software engineering involves multi-person construction of multi-version pro- 
grams. The IEEE 610.12 definition states that 


Definition 18.1 (Software Engineering) Software engineering is the application of 
a systematic, disciplined, quantifiable approach to the development, operation and 
maintenance of software, that is, the application of engineering to software and the 
study of such approaches. 


Software engineering includes: 


1. Methodologies to determine requirements, design, develop, implement and test 
software to meet customers’ needs. 

2. The philosophy of engineering, i.e. an engineering approach to developing 
software is adopted. That is, products are properly designed, developed, tested, 
with quality and safety properly addressed. 

3. Mathematics” may be employed to assist with the design and verification of 
software products. The level of mathematics to be employed will depend on the 
safety-critical nature of the product, as systematic peer reviews and testing are 
often sufficient. 

4. Sound project and quality management practices are employed. 


Software engineering requires the engineer to state precisely the requirements 
that the software product is to satisfy and then to produce designs that will meet 
these requirements. Engineers provide a precise description of the problem to be 
solved, they then proceed to producing a design and validating its correctness and 
finally the design is implemented and testing is performed to verify the correctness 


‘The process maturity models focus mainly on the management, engineering and organizational 
practices required in software engineering. The models focus on what needs to be done rather how 
it should be done. 

°There is no consensus at this time as to the appropriate role of mathematics in software 
engineering. The use of mathematics is invaluable in the safety critical and security critical fields 
as it provides an extra level of confidence in the correctness of the software. 
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Fig. 18.1 David Parnas 


of the implementation with respect to the requirements. The software requirements 
need to be unambiguous, and should clearly state what is and what is not required. 

Classical engineers produce the product design and then analyze their design for 
correctness. They use mathematics in their analysis, as this is the basis of con- 
firming that the specifications are met. The level of mathematics employed will 
depend on the particular application and calculations involved. The term ‘engineer’ 
is generally applied only to people who have attained the necessary education and 
competence to be called engineers, and who base their practice on mathematical and 
scientific principles. Often in computer science, the term engineer is employed 
rather loosely to refer to anyone who builds things, rather than to an individual with 
a core set of knowledge, experience and competence. 

Parnas® (Fig. 18.1) is a strong advocate of the classical engineering approach, 
and he argues that computer scientists should have the right education to apply 
scientific and mathematical principles to their work. This includes mathematics and 
design, to enable them to be able to build high-quality and safe products. Baber has 
argued [7] that “mathematics is the language of engineering”. He argues that stu- 
dents should be shown how to turn a specification into a program using 
mathematics. 

Parnas advocates a solid engineering approach to the teaching of mathematics 
with an emphasis on its application to developing and analyzing product designs. 
He argues that software engineers need education on engineering mathematics, 
specification and design, converting designs into programs, software inspections 
and testing. The education should enable the software engineer to produce 
well-designed programs that will correctly implement the requirements. 

He argues that software engineers have individual responsibilities as profes- 
sional engineers.’ They are responsible for designing and implementing 
high-quality and reliable software that is safe to use. They are also accountable for 


Parnas has made important contributions to software engineering including information hiding 
which is used in the object-oriented world. 

The concept of accountability is not new; indeed the ancient Babylonians employed a code of 
laws c. 1750 B.C. known as the Hammarabi Code. This code included the law that if a house 
collapsed and killed the owner then the builder of the house would be executed. 
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their own decisions and actions® and have a responsibility to object to decisions that 
violate professional standards. Professional engineers need to be honest about 
current capabilities, especially when asked to work on problems that have no 
appropriate technical solution. In other words, they should be honest and avoid 
accepting a contract for something that cannot be done. 

The licensing of a professional engineer provides confidence that the engineer 
has the right education and experience to build safe and reliable products. Pro- 
fessional engineers are required to follow rules of good practice, and to object when 
the rules are violated.” The professional engineering body is responsible for 
enforcing standards and certification. The term ‘engineer’ is a title that is awarded 
on merit, but it also places responsibilities on its holder. For a more detailed account 
of the professional responsibilities of software engineers see Chap. 23 of [8]. 

The approach used in current software engineering is to follow a well-defined 
software engineering process. The process includes activities such as project 
management, requirements gathering, requirements specification, architecture 
design, software design, coding and testing. Most companies use a set of templates 
for the various phases. The waterfall model [9] and spiral model [10] are popular 
software development lifecycles. 

The waterfall model (Fig. 18.2) starts with requirements, followed by specifi- 
cation, design, implementation and testing. It is typically used for projects where 
the requirements can be identified and it is often called the “V” life cycle model. 
The left-hand side of the “V” involves requirements, specification, design and 
coding and the right-hand side is concerned with unit tests, integration tests, system 
tests and acceptance testing. Each phase has entry and exit criteria that must be 
satisfied before the next phase commences. There are several variations of the 
waterfall model. 

The spiral model (Fig. 18.3) is useful where the requirements are not fully 
known at project initiation. There is an evolution of the requirements during 
development which proceeds in a number of spirals, with each spiral typically 
involves updates to the requirements, design, code, testing and a user review of the 
particular iteration or spiral. 


’However, it is unlikely that an individual programmer would be subject to litigation in the case of 
a flaw in a program causing damage or loss of life. A comprehensive disclaimer of responsibility 
for problems rather than a guarantee of quality accompany most software products. Software 
engineering is a team-based activity involving several engineers in various parts of the project, and 
it could be potentially difficult for an outside party to prove that the cause of a particular problem is 
due to the professional negligence of a particular software engineer, as there are many others 
involved in the process such as reviewers of documentation and code and the various test groups. 
Companies are more likely to be subject to litigation, as a company is legally responsible for the 
actions of their employees in the workplace, and the fact that a company is a financially richer 
entity than one of its employees. 

°Software companies that are following the CMMI or ISO 9000 will employ audits to verify that 
the rules and best practice have been followed. Auditors report their findings to management and 
the findings are addressed appropriately by the project team and affected individuals. 
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Fig. 18.2 Waterfall lifecycle : 
model (V-Model) Requirements Acceptance Testing 
pote System Testing 
Specification 
Design | | Integration Testing 
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| Code J/| Unit Testing | 


Fig. 18.3 SPIRAL lifecycle 
model 


The spiral is, in effect, a reusable prototype, and the customer examines the 
current iteration and provides feedback to the development team to be included in 
the next spiral. The approach is to partially implement the system. This leads to a 
better understanding of the requirements of the system and it then feeds into the 
next cycle in the spiral. The process repeats until the requirements and product are 
fully complete. 

There has been a growth of popularity among software developers in lightweight 
methodologies such as Agile. This is a software development methodology that 
claims to be more responsive to customer needs than traditional methods such as the 
waterfall model. The waterfall development model is similar to a wide and 
slow-moving value stream, and halfway through the project, 100% of the require- 
ments are typically 50% done. However, for agile development, 50% of require- 
ments are typically 100% done halfway through the project. 
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Ongoing changes to requirements are considered normal in the Agile world, and 
it is believed to be more realistic to change requirements regularly throughout the 
project rather than attempting to define all of the requirements at the start of the 
project. The methodology includes controls to manage changes to the requirements, 
and good communication and early regular feedback is an essential part of the 
process. 

A story may be a new feature or a modification to an existing feature. It is 
reduced to the minimum scope that can deliver business value, and a feature may 
give rise to several stories. Stories often build upon other stories and the entire 
software development lifecycle is employed for the implementation of each story. 
Stories are either done or not done, i.e. there is such thing as a story being 80% 
done. The story is complete only when it passes its acceptance tests, and software 
testing is described in more detail in [11]. For more details on Agile; see [6, 12]. 

The challenge in software engineering is to deliver high-quality software on time 
to customers. The Standish Group research (Fig. 18.4) on project cost overruns in 
the US during 1998, showed that 33% of projects are between 21 and 50% over- 
estimate, 18% are between 51 and 100% overestimate and 11% of projects are 
between 101 and 200% overestimate. 

The accurate estimation of project cost and effort are key challenges, and project 
managers need to determine how good their current estimation process actually is 
and to make improvements. Many companies today employ formal project man- 
agement methodologies such as Prince 2 or Project Management Professional 
(PMP). These methodologies allow projects to be rigorously managed and include 
processes for initiating a project, planning a project, executing a project, monitoring 
and controlling a project and closing a project. 

The Capability Maturity Model developed by the Software Engineering Institute 
(SEI) has become useful in software engineering. The SEI has collected empirical 
data to suggest that there is a close relationship between software process maturity 
and the quality and the reliability of the delivered software. The CMMI enables the 
organization to improve processes such as: 


Estimation Accuracy - 1998 @ Under 20% over 
021% to 50% over 
051% to 100% over 
01101% to 200% over 


2 


Estimation Accuracy Range 


bb 
Oo 


wo 
oO 


Frequency 
>N 
oo 


1 


Fig. 18.4 Standish group report—estimation accuracy 


304 18 Software Engineering Mathematics 


— Developing and managing requirements, 

— Design activities, 

— Configuration Management, 

— Selection and Management of Suppliers, 

— Planning and Managing projects, 

— Building quality into the product with peer reviews, 
— Performing rigorous testing, 

— Performing independent audits. 


The rest of this chapter is focused on mathematical techniques to support soft- 
ware engineering to improve software quality, and the chapter concludes with a 
short discussion on software inspections and testing and process maturity models. 
For a more detailed account of software engineering see [13]. 


18.3 Early Software Engineering Mathematics 


Robert Floyd was born in New York, in 1936, and he did pioneering work on 
software engineering from the 1960s. He made important contributions to the 
theory of parsing; the semantics of programming languages, program verification 
and methodologies for the creation of efficient and reliable software (Fig. 18.5). 

Mathematics and Computer Science were regarded as two completely separate 
disciplines in the 1960s, and software development was based on the assumption 
that the completed code would always contain defects. It was, therefore, better and 
more productive to write the code as quickly as possible, and to then perform 
debugging to find the defects. Programmers then corrected the defects, made pat- 
ches and retested and found more defects. This continued until they could no longer 
find defects. Of course, there was always the danger that defects remained in the 
code that could give rise to software failures. 


Fig. 18.5 Robert Floyd 
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Floyd believed that there was a way to construct rigorous proof of the correct- 
ness of the programs using mathematics. He showed that mathematics could be 
used for program verification, and he introduced the concept of assertions that 
provided a way to verify the correctness of programs. 

Flowcharts were employed in the 1960s to explain the sequence of basic steps 
for computer programs. Floyd’s insight was to build upon flowcharts and to apply 
an invariant assertion to each branch in the flowchart. These assertions state the 
essential relations that exist between the variables at that point in the flow chart. An 
example relation is ‘R= Z>0, X= 1, Y=0’. He devised a general flowchart 
language to apply his method to programming languages. The language essentially 
contains boxes linked by the flow of control arrows [14]. 

Consider the assertion Q that is true on entry to a branch where the condition at 
the branch is P. Then, the assertion on exit from the branch is Q A —P if P is false 
and Q /\ P otherwise (Fig. 18.6). 

The use of assertions may be employed in an assignment statement. Suppose 
x represents a variable and v represents a vector consisting of all the variables in the 
program. Suppose f(x, v) represents a function or expression of x and the other 
program variables represented by the vector v. Suppose the assertion S(f(x, v), v) is 
true before the assignment x = f(x, v). Then the assertion S(x, v) is true after the 
assignment (Fig. 18.7). This is given by 

Floyd used flowchart symbols to represent entry and exit to the flowchart. This 
included entry and exit assertions to describe the program’s entry and exit 
conditions. 


Fig. 18.6 Branch assertions in flowcharts 
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Fig. 18.7 Assignment assertions in flowcharts 
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Floyd’s technique showed how a computer program is a sequence of logical 
assertions. Each assertion is true whenever control passes to it, and statements 
appear between the assertions. The initial assertion states the conditions that must 
be true for the execution of the program to take place, and the exit assertion 
essentially describes what must be true when the program terminates. 

His key insight was the recognition that if it can be shown that the assertion 
immediately following each step is a consequence of the assertion immediately 
preceding it, then the assertion at the end of the program will be true, provided the 
appropriate assertion was true at the beginning of the program. 

He published an influential paper, “Assigning Meanings to Programs’, in 1967 
[14], and this paper influenced Hoare’s work on preconditions and post-conditions 
leading to Hoare logic [15]. Floyd’s paper also presented a formal grammar for 
flowcharts, together with rigorous methods for verifying the effects of basic actions 
like assignments. 

Hoare logic is a formal system of logic used for programming semantics and for 
program verification. It was developed by C. A. R. Hoare (Fig. 18.8)., and was 
originally published in Hoare’s 1969 paper “An axiomatic basis for computer 
programming” [15]. Hoare and others have subsequently refined it, and it provides 
a logical methodology for precise reasoning about the correctness of computer 
programs. 

Hoare was influenced by Floyd’s 1967 paper that applied assertions to flow- 
charts, and he recognized that this provided an effective method for proving the 
correctness of programs. He built upon Floyd’s approach to cover the familiar 
constructs of high-level programming languages. 

This led to the axiomatic approach to defining the semantics of every statement 
in a programming language (see Chap. 12), and the approach consists of axioms 
and proof rules. He introduced what has become known as the Hoare triple, and this 
describes how the execution of a fragment of code changes the state. A Hoare triple 
is of the form 


P{Q}R 


Fig. 18.8 C.A.R Hoare 
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where P and R are assertions and Q is a program or command. The predicate P is 
called the precondition and the predicate R is called the postcondition. 


Definition 18.2 (Partial Correctness) The meaning of the Hoare triple above is that 
whenever the predicate P holds of the state before the execution of the command or 
program Q, then the predicate R will hold after the execution of Q. The brackets 
indicate partial correctness as if Q does not terminate then R can be any predicate. 
R may be chosen to be false to express that Q does not terminate. 


Total correctness requires Q to terminate, and at termination R is true. Termi- 
nation needs to be proved separately. Hoare logic includes axioms and rules of 
inference rules for the constructs of imperative programming language. 

Hoare and Dijkstra were of the view that the starting point of a program should 
always be the specification, and that the proof of the correctness of the program 
should be developed along with the program itself. 

That is, the starting point is the mathematical specification of what a program is 
to do, and mathematical transformations are applied to the specification until it is 
turned into a program that can be executed. The resulting program is then known to 
be correct by construction. 


18.4 Mathematics in Software Engineering 


Mathematics plays a key role in classical engineering to assist with the design and 
verification of software products. It is, therefore, reasonable to apply appropriate 
mathematics in software engineering (especially for safety and security-critical 
systems) to assure that the delivered systems conform to the requirements. The 
extent to which mathematics should be used is controversial with strong views in 
both camps. In many cases, peer reviews and testing will be sufficient to build 
quality into the software product. In other cases, and especially with safety and 
security-critical applications, it is desirable to have the extra assurance that may be 
provided with mathematical techniques. 

Mathematics allows a rigorous analysis to take place and avoids an over-reliance 
on intuition. The emphasis is on applying mathematics to solve practical problems 
and to develop products that are fit for use. Engineers are taught how to apply 
mathematics in their work, and the emphasis is always on the application of 
mathematics to solve practical problems. 

Classical mathematics may be applied to software engineering and specialized 
mathematical methods and notations have also been developed. The classical 
mathematics employed includes sets, relations, functions, logic, graph theory, 
automata theory, matrix theory, probability and statistics, calculus and matrix 
theory. Specialized formal specification languages such as Z and VDM have been 
developed, and these allow the requirements to be formally specified in precise 
mathematical language. 


308 18 Software Engineering Mathematics 


The term ‘formal methods’ refers to various mathematical techniques used in the 
software field for the specification and formal development of software. Formal 
methods consist of formal specification languages or notations and employ a col- 
lection of tools to support the syntax checking of the specification, as well as the 
proof of properties about the specification. The term ‘formal methods’ is used to 
describe a formal specification language and a method for the design and imple- 
mentation of computer systems. 

The mathematical analysis of the formal specification allows questions to be 
asked about what the system does, and these questions may be answered inde- 
pendently of the implementation. Mathematical notation is precise, and this helps to 
avoid the problem of ambiguity inherent in a natural language description of a 
system. The formal specification may be used to promote a common understanding 
for all stakeholders. 

Formal methods have been applied to a diverse range of applications, including 
the safety-critical field, security-critical field, the railway sector, the nuclear field, 
microprocessor verification, the specification of standards and the specification and 
verification of programs. 

There are various tools to support formal methods including syntax checkers, 
specialized editors, tools to support refinement, automated code generators, theorem 
provers and specification animation tools. Formal methods need to mature further 
before they will be used in mainstream software engineering, and they are described 
in more detail in Chap. 20. 


18.5 Software Inspections and Testing 


Software inspections play an important role in building quality into software 
products. The Fagan Inspection Methodology was developed by Michael Fagan at 
IBM in the mid-1970s [16]. It is a seven-step process that identifies and removes 
defects in work products. The Fagan methodology mandates that requirement 
documents, design documents, source code and test plans are all formally inspected. 

There are several roles defined in the process including the moderator who 
chairs the inspection; the reader who reads or paraphrases the particular deliver- 
able; the author who is the creator of the deliverable; and the tester who is con- 
cerned with the testing viewpoint. 

The inspection process will consider whether a design is correct with respect to 
the requirements and whether the source code is correct with respect to the design. 
There are several stages in the Fagan inspection process, including planning, 
overview, preparation, inspection, process improvement, rework and follow-up. 

Software testing plays a key role in verifying that a software product is of high 
quality and conforms to the customer’s quality expectations. Testing is both a 
constructive activity in that it is verifying the correctness of functionality, and it is 
also a destructive activity in that the objective is to find as many defects as possible 
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in the software. The testing verifies that the requirements are correctly implemented 
as well as identifying whether any defects are present in the software product. 

There are various types of testing such as unit testing, integration testing, system 
testing, performance testing, usability testing, regression testing, and customer 
acceptance testing. The testing needs to be planned and test cases prepared and 
executed. The results of testing are reported and any issues corrected and re-tested. 
The test cases will need to be appropriate to verify the correctness of the software. 
Software inspection and testing are described in more detail in [6]. 


18.6 Process Maturity Models 


The Software Engineering Institute (SEI) developed the Capability Maturity Model 
(CMM) in the early 1990s as a framework to help software organizations to 
improve their software process maturity, and to implement best practice in software 
and systems engineering. The SEI believes that there is a close relationship between 
the maturity of software processes and the quality of the delivered software product. 

The CMM applied the ideas of Deming [17], Juran [18], and Crosby [19] to the 
software field. These quality gurus were influential in transforming manufacturing 
companies with quality problems into effective quality-driven organizations with a 
reduced cost of poor quality. 

Watt Humphries (Fig. 18.9) did early work on software process improvement at 
IBM [20]. He moved to the SEI in the late 1980s and the first version of the CMM 
was released in 1991. It is now called the Capability Maturity Model Integration 
(CMMI®) [21]. 

The CMMI consists of five maturity levels with each maturity level (except level 
one) consisting of several process areas. Each process area consists of a set of goals 
that are implemented by practices related to that process area leading to an effective 
process. 

The emphasis on level two of the CMMI is on maturing management practices 
such as project management, requirements management, configuration management 
and so on. The emphasis on level three of the CMMI is to mature engineering and 


Fig. 18.9 Watts Humphrey. 
Courtesy of Watts Humphrey 
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organization practices. This maturity level includes peer reviews and testing, 
requirements development, software design and implementation practices and so 
on. Level four is concerned with ensuring that key processes are performing within 
strict quantitative limits, and adjusting processes where necessary to perform within 
these defined limits. Level five is concerned with continuous process improvement, 
which is quantitatively verified. 

The CMMI allows organizations to benchmark themselves against other similar 
organizations. This is done by appraisals conducted by an authorized SCAMPI lead 
appraiser. The results of a SCAMPI appraisal are generally reported back to the 
SEL, and there is a strict qualification process to become an authorized lead 
appraiser. An appraisal is useful in verifying that an organization has improved, and 
it enables the organization to prioritize improvements for the next improvement 
cycle. 


18.7 Review Questions 


1. What is software engineering? Describe the difference between classical 
engineers and software engineers 

2. Describe the “software crisis” of the late 1960s that led to the first 
software engineering conference in 1968 

3. Discuss the Standish Research Report and the level of success of IT 
projects today. In your view is there a crisis in software engineering 
today? Give reasons for your answer 

4. Discuss what the role of mathematics should be in current software 
engineering 

5. Describe the waterfall and spiral lifecycles. What are the similarities and 

differences between them? 

Discuss the contributions of Floyd and Hoare 

Explain the difference between partial correctness and total correctness 

What are formal methods? 

Discuss the process maturity models (including the CMMI). What are 

their advantages and disadvantages? 

10. Discuss how software inspections and testing can assist in the delivery of 
high-quality software 
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18.8 Summary 


This chapter presented a short account of some important developments in software 
engineering. Its birth was at the Garmisch conference, in 1968, and it was recog- 
nized that there was a crisis in the software field, and a need for sound method- 
ologies to design, develop and maintain software to meet customer needs. 

Classical engineering has a successful track record in building high-quality 
products that are safe for the public to use. It is, therefore, natural to consider using 
an engineering approach to developing software, and this involves identifying the 
customer requirements, carrying out a rigorous design to meet the requirements, 
developing and coding a solution to meet the design and conducting appropriate 
inspections and testing to verify the correctness of the solution. 

Mathematics plays a key role in classical engineering to assist with the design 
and verification of software products. It is, therefore, reasonable to apply appro- 
priate mathematics in software engineering (especially for safety-critical systems) to 
ensure that the delivered system conforms to the requirements. The extent to which 
mathematics should be used is controversial with strong views in both camps. 

There is a lot more to the successful delivery of a project than just the use of 
mathematics or peer reviews and testing. Sound project management and quality 
management practices are essential, as a project that is not properly managed will 
suffer from schedule, budget or cost overruns as well as problems with quality. 

Maturity models such as the CMMI can assist organizations in maturing key 
management and engineering practices and may help companies in their goals to 
deliver high-quality software systems that are consistently delivered on time and 
budget. 
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Software Reliability and Dependability 


19.1. Introduction 


This chapter gives an introduction to the important area of software reliability and 
dependability, and it discusses important topics in software engineering such as 
software reliability, software availability, software reliability models, the Clean- 
room methodology, dependability and its various dimensions, security engineering 
and safety-critical systems. 

Software reliability is the probability that the program works without failure for 
a period of time, and it is usually expressed as the mean time to failure. It is 
different from hardware reliability, which is characterized by components that 
physically wear out over time, whereas software is an intellectual creation and is 
intangible and does not physically wear out as such. In other words, software 
failures are due to design and implementation errors as the software is either correct 
or incorrect when it is designed and developed, and it does not physically deteri- 
orate with time. 

Harlan Mills and others at IBM developed the Cleanroom approach to software 
development, and the process is described in Sect. 19.2.2 and in more detail in [1]. 
It involves the application of statistical techniques to calculate a software reliability 
measure based on the expected usage of the software.’ This involves executing tests 
chosen from the population of all possible uses of the software in accordance with 
the probability of its expected use. Statistical usage testing has been shown to be 
more effective in finding defects that lead to failure than coverage testing. 


'The expected usage of the software (or operational profile) is a quantitative characterization 
(usually based on probability) of how the system will be used. 
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Models are simplifications of reality and a good model allows accurate predic- 
tions of future behaviour to be made. A model is judged effective if there is good 
empirical evidence to support it, and a good software reliability model will have 
good theoretical foundations and realistic assumptions. The extent to which the 
software reliability model can be trusted depends on the accuracy of its predictions, 
and empirical data will need to be gathered to judge its effectiveness. A good 
software reliability model will give good predictions of the reliability of the 
software. 

It is essential that software that is widely used is dependable, which means that 
the software is available whenever required and that it operates safely and reliably 
without any adverse side effects (e.g. the software problems with the Therac-25 
radiography machine led to several patients receiving massive overdoses in radia- 
tion in the mid-1980s leading to serious injury and death of several patients). 

Today, billions of computers are connected to the Internet, and this has led to a 
growth in attacks on computers. It is essential that computer security is carefully 
considered, and developers need to be aware of the threats facing a system and 
techniques to eliminate them. The developers need to be able to develop secure 
systems that are able to deal with and recover from external attacks. 


19.2 Software Reliability 


The design and development of high-quality software has become increasingly 
important for society. The hardware field has been very successful in developing 
sound reliability models, which allow useful and accurate predictions of how long a 
hardware component (or product) will function. This has led to a growing interest in 
the software field in the development of a sound software reliability model. Such a 
model would provide a sound mechanism to predict the reliability of the software 
prior to its deployment at the customer site, as well as confidence that the software 
is fit for purpose and safe to use. 


Definition 19.1 (Software Reliability) Software reliability is the probability that 
the program works without failure for a specified length of time, and it is a state- 
ment of the future behaviour of the software. It is generally expressed in terms of 
the mean time to failure (MTTF) or the mean time between failure (MTBF). 


Statistical sampling techniques are often employed to predict the reliability of 
hardware, as it is not feasible to test all items in a production environment. The 
quality of the sample is then used to make inferences on the quality of the entire 
population, and this approach is effective in manufacturing environments where 
variations in the manufacturing process often lead to defects in the physical 
products. 
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There are similarities and differences between hardware and software reliability. 
A hardware failure generally arises due to a component wearing out due to its age, 
and often a replacement component is required. Many hardware components are 
expected to last for a certain period of time, and the variation in the failure rate of a 
hardware component is often due to variations in the manufacturing process, and to 
the operating environment of the component. Good hardware reliability predictors 
have been developed, and each hardware component has an expected mean time to 
failure. The reliability of a product may then be determined from the reliability of 
the individual components. 

Software is an intellectual undertaking involving a team of designers and pro- 
grammers. It does not physically wear out as such, and software failures manifest 
themselves from particular user inputs. Each copy of the software code is identical, 
and the software code is either correct or incorrect. That is, software failures are due 
to design and implementation errors, rather than to the software physically wearing 
out over time. The software community has not yet developed a sound software 
reliability predictor model. 

The software population to be sampled consists of all possible execution paths of 
the software, and since this is potentially infinite it is generally not possible to 
perform exhaustive testing. The way in which the software is used (i.e. the inputs 
entered by the users) will impact upon its perceived reliability. Let Ir represent the 
fault set of inputs (i.e. iy € Ipif and only if the input of i by the user leads to failure). 
The randomness of the time to software failure is due to the unpredictability in the 
selection of an input iy € Ip. It may be that the elements in I are inputs that are 
rarely used, and therefore, the software will be perceived as being reliable. 

Statistical usage testing may be used to make predictions on the future perfor- 
mance and reliability of the software. This requires an understanding of the 
expected usage profile of the system, as well as the population of all possible usages 
of the software. The sampling is done in accordance with the expected usage 
profile, and a software reliability measure is calculated. 


19.2.1 Software Reliability and Defects 


The release of an unreliable software product may result in damage to property or 
injury (including loss of life) to a third party. Consequently, companies need to be 
confident that their software products are fit for use prior to their release. The 
project team needs to conduct extensive inspections and testing of the software, as 
well as considering all associated risks prior to its release. 

Objective product quality criteria may be set (e.g. 100% of tests performed and 
passed) that must be satisfied prior to the release of the product. This provides a 
degree of confidence that the software has the desired quality, and is fit for its 
purpose. However, these results are historical in the sense that they are a statement 
of past and present quality. The question is whether the past behavior and perfor- 
mance provides a sound indication of the future behaviour. 
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Table 19.1 Adam’s 1984 study of software failures of IBM PRODUCTS 


Rare Frequent 

1 2 3 4 5 6 fi 8 
MITTF (years) —_ 5,000 1,580 500 158 50 15.8 5 1.58 
Avg % Fixes 33.4 28.2 18.7 10.6 a 2s) 1.0 0.4 


Prob Failure 0.008 0.021 0.044 0.079 0.123 0.187 0.237 0.300 


Software reliability models are an attempt to predict the future reliability of the 
software and to assist in deciding on whether the software is ready for release. 
A defect does not always result in a failure, as it may occur on a rarely used 
execution path. Studies indicate that many observed failures arise from a small 
proportion of the existing defects. 

Adam’s 1984 case study of defects in IBM software [2] indicate that over 33% 
of the defects led to an observed failure with mean time to failure greater than 
5000 years, whereas less than 2% of defects led to an observed failure with a mean 
time to failure of less than 5 years. This suggests (if these results may be gener- 
alized to the wider software field) that a small proportion of defects often lead to 
almost all of the observed failures (Table 19.1). 

The analysis shows that 61.6% of all fixes (Group | and 2) were for failures that 
will be observed less than once in 1580 years of expected use and that these 
constitute only 2.9% of the failures observed by typical users. On the other hand, 
groups 7 and 8 constitute 53.7% of the failures observed by typical users and only 
1.4% of fixes. 

This case study indicates that coverage testing is not cost effective in increasing 
MTTF. Usage testing, in contrast, would allocate 53.7% of the test effort to fixes 
that will occur 53.7% of the time for a typical user. Harlan Mills has argued [3] that 
the data in the table shows that usage testing is 21 times more effective than 
coverage testing. 

There is a need to be careful with reliability growth models, as there is no 
tangible growth in reliability unless the corrected defects are likely to manifest 
themselves as a failure.” Many existing software reliability growth models assume 
that all remaining defects in the software have an equal probability of failure and 
that the correction of a defect leads to an increase in software reliability. These 
assumptions are questionable. 

The defect count and defect density may be poor predictors of operational 
reliability, and an emphasis on removing a large number of defects from the 
software may not be sufficient to achieve high reliability. 

The correction of defects in the software leads to a newer version of the soft- 
ware, and many software reliability models assume reliability growth, i.e. the new 
version is more reliable than the older version as several identified defects have 


*We are assuming that the defect has been corrected perfectly with no new defects introduced by 
the changes made. 
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Table 19.2 New and old 
version of the software 


Similarities and differences between new/old version 


¢ The new version of the software is identical to the previous 
version except that the identified defects have been 
corrected 


¢ The new version of the software is identical to the previous 
version, except that the identified defects have been 
corrected, but the developers have introduced some new 
defects 


¢ No assumptions can be made about the behaviour of the 
new version of the software until further data is obtained 


been corrected. However, in some sectors such as the safety-critical field, the view 
is that the new version of a program is a new entity and that no inferences may be 
drawn until further investigation has been done. There are a number of ways to 
interpret the relationship between the new version of the software and the older 
version (Table 19.2). 

The safety-critical industry (e.g. the nuclear power industry) takes the conser- 
vative viewpoint that any change to a program creates a new program. The new 
program is, therefore, required to demonstrate its reliability, and so extensive testing 
needs to be performed. 


19.2.2 Cleanroom Methodology 


Harlan Mills and others at IBM developed the Cleanroom methodology as a way to 
develop high-quality software [3]. Cleanroom helps to ensure that the software is 
released only when it has achieved the desired quality level, and the probability of 
zero defects is very high. 

The way in which the software is used will impact its perceived quality and 
reliability. Failures will manifest themselves on certain input sequences, and as the 
input sequences will vary among users, the result will be different perceptions of the 
reliability of the software among the users. The knowledge of how the software will 
be used allows the software testing to focus on verifying the correctness of common 
everyday tasks carried out by users. 

Therefore, it is important to determine the operational profile of the users to enable 
effective software testing to be performed. This may be difficult to determine and 
could change over time, as users may potentially change their behaviour as their needs 
evolve. The determination of the operational profile involves identifying the common 
operations to be performed, and the probability of each operation being performed. 

Cleanroom employs statistical usage testing rather than coverage testing, and 
this involves executing tests chosen from the population of all possible uses of the 
software in accordance with the probability of its expected use. The software 
reliability measure is calculated by statistical techniques based on the expected 
usage of the software, and Cleanroom provides a certified mean time to failure of 
the software. 
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Table 19.3 Cleanroom results in IBM 


Project Results 


Flight Control project (1987) 33KLOC Completed ahead of schedule 
Error-fix effort reduced by factor of five 
2.5 errors KLOC before any execution 


Commercial Product (1988) Deployment failures of 0.1/KLOC 
Certification testing failures 3.4/KLOC 
Productivity 740 LOC/month 


Satellite Control (1989) 80 KLOC 50% improvement in quality 

(partial Cleanroom) Certification testing failures of 3.3/KLOC 
Productivity 780 LOC/month 
80% improvement in productivity 


Research project (1990) 12 KLOC Certified to 0.9978 with 989 test cases 


Coverage testing involves designing tests that cover every path through the 
program, and this type of testing is as likely to find a rare execution failure as well 
as a frequent execution failure. However, it is essential to find failures that occur on 
frequently used parts of the system. 

The advantage of usage testing (that matches the actual execution profile of the 
software) is that it has a better chance of finding execution failures on frequently 
used parts of the system. This helps to maximize the expected mean time to failure 
of the software. 

The Cleanroom software development process and calculation of the software 
reliability measure is described in [1], and the Cleanroom development process 
enables engineers to deliver high-quality software on time and on budget. Some of 
the benefits of the use of Cleanroom on projects at IBM are described in [3] and 
summarized in Table 19.3. 


19.2.3 Software Reliability Models 


System availability is the percentage of time that the software system is running 
without downtime, and robust systems will generally aim to achieve 5-nines 
availability (i.e. 99.999% availability). This is equivalent to approximately 5 min of 
downtime (including planned/unplanned outages) per year. The availability of a 
system is measured by its performance when a subsystem fails, and its ability to 
resume service in a state close to the original state. A fault-tolerant system continues 
to operate correctly (possibly at a reduced level) after some part of the system fails, 
and it aims to achieve 100% availability. 

A model is judged effective if there is good empirical evidence to support it, and 
more accurate models are sought to replace inadequate models. Models are often 
modified (or replaced) over time, as further facts and observations lead to aberra- 
tions that cannot be explained with the current model. A good software reliability 
model will have the following characteristics (Table 19.4). 
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Table 19.4 Characteristics 
of good software reliability 
model 


Characteristics of good software reliability model 


Good theoretical foundation 

Realistic assumptions 

Good empirical support 

As simple as possible (Ockham’s Razor) 
Trustworthy and accurate 


The underlying mathematics used in the calculation of software reliability (i.e. 
probability and statistics) is discussed in Chaps. 22 and 23. There are several 
existing software reliability predictor models employed (Table 19.5) with varying 
degrees of success. Some of these models just compute defect counts rather than 
estimating software reliability in terms of mean time to failure. They may be 
categorized into: 


e Size and Complexity Metrics 
These are used to predict the number of defects that a system will reveal in oper- 
ation or testing. 


¢ Operational Usage Profile 

These predict failure rates based on the expected operational usage profile of the 
system. The number of failures encountered is determined and the software relia- 
bility predicted (e.g. Cleanroom and its prediction of the MTTF). 


¢ Quality of the Development Process 
These predict failure rates based on the process maturity of the software develop- 
ment process in the organization (e.g. CMMI maturity). 


The extent to which the software reliability model can be trusted depends on the 
accuracy of its predictions, and empirical data will need to be gathered to make a 
judgement. It may be acceptable to have a little inaccuracy during the early stages 
of prediction, provided the predictions of operational reliability are close to the 
observations. A model that gives overly optimistic results is termed ‘optimistic’, 
whereas a model that gives overly pessimistic results is termed ‘pessimistic’. 

The assumptions in the reliability model need to be examined to determine 
whether they are realistic. Several software reliability models have questionable 
assumptions such as: 


All defects are corrected perfectly. 

Defects are independent of one another. 

Failure rate decreases as defects are corrected. 

Each fault contributes the same amount to the failure rate. 
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Table 19.5 Software reliability models 


Model Description Comments 
Jelinski/Moranda The failure rate is a Poisson process* Assumes defects corrected perfectly 
Model and is proportional to the current defect and no new defects are introduced 


content of program. The initial defect Assumes each fault contributes the 
count is N; the initial failure rate is Ng; same amount to the failure rate 

it decreases to (N-1)¢ after the first fault 

is detected and eliminated, and so on. 

The constant @ is termed the 

proportionality constant 


Littlewood/Verrall Successive execution time between Does not assume perfect correction of 
Model failures is independent exponentially defects 

distributed random variables.* 

Software failures are the result of the 

particular inputs and faults introduced 

from the correction of defects 


Seeding and This is analogous to estimating the fish Estimate of the total number of 

Tagging population of a lake (Mills). A known defects in the software but not a 
number of defects are inserted into a software reliability predictor 
software program, and the proportion of Assumes all faults equally likely to be 
these identified during testing found and introduced faults 
determined representative of existing 


Another approach (Hyman) is to regard 

the defects found by one tester as tagged, 

and then to determine the proportion of 

tagged defects found by a 2nd 

independent tester 
Generalized The number of failures observed in ith Assumes faults removed perfectly at 
Poisson Model time interval t; has a Poisson distribution end of time interval 

with mean b(N — M,-1) 1” where N is 

the initial number of faults; Mj-; is the 

total number of faults removed up to the 

end of the (i — 1)th time interval; and 

is the proportionality constant 


19.3. Dependability 


Software is ubiquitous and is important to all sections of society, and so it is 
essential that software that is widely used is dependable (or trustworthy). In other 
words, the software should be available whenever required, as well as operating 
properly, safely and reliably, without any adverse side effects or security concerns. 
It is essential that the software used in systems in the safety-critical and 


3The Poisson process is a widely used counting process, especially in counting the occurrence of 
certain events that appear to happen at a certain rate but at random. A Poisson random variable is 
of the form P{X =i} =e * NM /il. 

“The exponential distribution is used to model the time between the occurrence of events in an 
interval of time. The exponential density function is given by f(x) = re, 
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Table 19.6 Dimensions of dependability 


Dimension Description 

Availability The system is available for use at any time 

Reliability The system operates correctly and is trustworthy 

Safety The system does not injure people or damage the environment 
Security The system prevents unauthorized intrusions 


security-critical fields is dependable, as the consequence of failure (e.g. the failure 
of a nuclear power plant) could be massive damage leading to loss of life or 
endangering the lives of the public. 

Dependability engineering is concerned with techniques to improve the 
dependability of systems, and it involves the use of a rigorous design and devel- 
opment process to minimize the number of defects in the software. A dependable 
system is generally designed for fault tolerance, where the system can deal with 
(and recover from) faults that occur during software execution. Such a system needs 
to be secure and able to protect itself from accidental or deliberate external attacks. 
Table 19.6 lists several dimensions of dependability: 

Modern software systems are subject to attack by malicious software such as 
viruses that change the behaviour of the software, or corrupt data causing the 
system to become unreliable. Other malicious attacks include a denial of service 
attack that negatively impacts the system’s availability. 

The design and development of dependable software needs to include protection 
measures that protect against external attacks that could compromise the availability 
and security of the system. Further, a dependable system needs to include recovery 
mechanisms to enable normal service to be restored as quickly as possible fol- 
lowing an attack. 

Dependability engineering is concerned with techniques to improve the 
dependability of systems, and in designing dependable systems. A dependable 
system will generally be developed using an explicitly defined repeatable process, 
and it may employ redundancy (spare capacity) and diversity (different types) to 
achieve reliability. 

There is a trade-off between dependability and the performance of the system, as 
dependable systems often need to carry out extra checks to monitor themselves and 
to check for erroneous states, and to recover from faults before failure occurs. This 
inevitably leads to increased costs in the design and development of dependable 
systems. 

Software availability is the percentage of the time that the software system is 
running, and is a measure of the uptime/downtime of the software during a par- 
ticular time period. The downtime refers to a period of time when the software is 
unavailable for use (including planned and unplanned outages), and many com- 
panies aim to develop software that is available for use 99.999% of the time in the 
year (i.e. a downtime of less than 5 min per annum). This goal is known as five 
nines, and it is a common goal in the telecommunications sector. 
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Safety-critical systems are systems where it is essential that the system is safe for 
the public, and that people or the environment are not harmed in the event of system 
failure. These include aircraft control systems and process control systems for 
chemical and nuclear power plants. The failure of a safety-critical system could in 
some situations lead to loss of life or serious economic damage. 

Formal methods are discussed in Chap. 20, and they provide a precise way of 
specifying the requirements of the proposed system and demonstrating (using 
mathematics) that key properties are satisfied in the formal specification. Further, 
they may be used to show that the implemented program satisfies its specification. 
The use of formal methods generally leads to increased confidence in the correct- 
ness of safety-critical and security-critical systems. 

The security of the system refers to its ability to protect itself from accidental or 
deliberate external attacks, which are common today since most computers are 
networked and connected to the Internet. There are various security threats in any 
networked system including threats to the confidentiality and integrity of the system 
and its data, and threats to the availability of the system. 

Therefore, controls are required to enhance security and to ensure that attacks are 
unsuccessful. Encryption is one way to reduce system vulnerability, as encrypted 
data is unreadable to the attacker. There may be controls that detect and repel 
attacks, and these controls are used to monitor the system and to take action to shut 
down parts of the system or restrict access in the event of an attack. There may be 
controls that limit exposure (e.g. insurance policies and automated backup strate- 
gies) that allow recovery from the problems introduced. 

It is important to have a reasonable level of security as otherwise all of the other 
dimensions of dependability (reliability, availability and safety) are compromised. 
Security loopholes may be introduced in the development of the system, and so care 
needs to be taken to prevent hackers from exploiting security vulnerabilities. 

Risk analysis plays a key role in the specification of security and dependability 
requirements, and this involves identifying risks that can result in serious incidents. 
This leads to the generation of specific security requirements as part of the system 
requirements to ensure that these risks do not materialize, or if they do materialize 
then serious incidents will not materialize. 


19.4 Computer Security 


The introduction of the world wide web in the early 1990s transformed the world of 
computing, and it led inexorably to more and more computers being connected to 
the Internet. This has subsequently led to an explosive growth in attacks on com- 
puters and systems, as hackers and malicious software seek to exploit known 
security vulnerabilities. It is, therefore, essential to develop secure systems that can 
deal with and recover from such external attacks. 
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Hackers will often attempt to steal confidential data and disrupt the services 
being offered by a system. Security engineering is concerned with the development 
of systems that can prevent such malicious attacks and recover data from them. It 
has become an important part of the software and system engineering, and software 
developers need to be aware of the threats faced by a system, and develop solutions 
to eliminate them. 

Hackers may probe parts of the system for weaknesses, and system vulnera- 
bilities may lead to attackers gaining unauthorized access to the system. There is a 
need to conduct a risk assessment of the security threats facing a system early in the 
software development process, and this will lead to several security requirements 
for the system. 

The system needs to be designed for security, as it is difficult to add security after 
it has been implemented. Security loopholes may be introduced in the development 
of the system, and so care needs to be taken to prevent these as well as preventing 
hackers from exploiting security vulnerabilities. There may be controls that detect 
and repel attacks, and these monitor the system and take appropriate action to 
restrict access in the event of an attack. Users need to be educated on cyber hygiene, 
as often attackers gain entry to a system after a user clicks on what appears to be a 
genuine mail but which launches malware into the system. 

The choice of architecture and how the system is organized is fundamental to the 
security of the system, and different types of systems will require different technical 
solutions to provide an acceptable level of security to its users. There following 
guidelines for designing secure systems are described in [4]: 


— Security decisions should be based on the security policy. 

— A security-critical system should fail securely. 

— A secure system should be designed for recoverability. 

— A balance is needed between security and usability. 

— A single point of failure should be avoided. 

— A log of user actions should be maintained. 

— Redundancy and diversity should be employed. 

— Organization of information in the system into compartments. 


It is important to have a good level of security, as otherwise all of the other 
dimensions of dependability are compromised. 


19.5 System Availability 


System availability is the percentage of time that the software system is running 
without downtime, and robust systems will generally aim to achieve 5-nines 
availability (i.e. 99.999% availability). This is equivalent to approximately 5 min of 
downtime (including planned/unplanned outages) per year. The availability of a 
system is measured by its performance when a subsystem fails, and its ability to 
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resume service in a State close to the original state. A fault-tolerant system continues 
to operate correctly (possibly at a reduced level) after some part of the system fails, 
and it aims to achieve 100% availability. 

System availability and software reliability are related, with availability mea- 
suring the percentage of time that the system is operational, and reliability mea- 
suring the probability of failure-free operation over a period of time. The 
consequence of a system failure may be to freeze or crash the system, and system 
availability is measured by how long it takes to recover and restart after a failure. 
A system may be unreliable and yet have good availability metrics (fast restart after 
failure), or it may be highly reliable with poor availability metrics (taking a long 
time to recover after a failure). 

Software that satisfies strict availability constraints is usually reliable. The 
downtime generally includes the time needed for activities such as rebooting a 
machine, upgrading to a new version of software planned and unplanned outages. 


19.6 Safety-Critical Systems 


A safety-critical system is a system whose failure could result in significant eco- 
nomic damage or loss of life. There are many examples of safety-critical systems 
including aircraft flight control systems and missile systems. It is, therefore, 
essential to employ rigorous processes in their design and development, and testing 
alone is usually insufficient in verifying their correctness. 

The safety-critical industry takes the view that any change to safety-critical 
software creates a new program. The new program is, therefore, required to 
demonstrate that it is reliable and safe to use, and so extensive testing needs to be 
performed. Other techniques such as formal verification and model checking may 
be employed to provide an extra level of assurance in its correctness. 

Safety-critical systems need to be dependable and available for use whenever 
required, and this software must operate correctly and reliably without any adverse 
side effects. The consequence of failure (e.g. the failure of a weapons system) could 
be massive damage, leading to loss of life or endangering the lives of the public. 

Safety-critical systems are generally designed for fault tolerance, where the 
system can deal with (and recover from) faults that occur during execution. Fault 
tolerance is achieved by anticipating exceptional events, and in designing the 
system to handle them. A fault-tolerant system is designed to fail safely, and 
programs are designed to continue working (possibly at a reduced level of per- 
formance) rather than crashing after the occurrence of an error or exception. Many 
fault-tolerant systems mirror all operations, where each operation is performed on 
two or more duplicate systems, and so if one fails then the other system can take 
over. 
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The development of a safety-critical system needs to be rigorous, and subject to 
strict quality assurance to ensure that the system is safe to use and that the public 
will not be in danger. This involves rigorous design and development processes to 
minimize the number of defects in the software, as well as comprehensive testing to 
verify its correctness. Formal methods are often employed in the development of 
safety-critical systems (see Chap. 20). 


19.7 Review Questions 


Explain the difference between software reliability and system availability? 
What is software dependability? 

Explain the significance of Adam’s 1984 study of failures at IBM? 
Describe the Cleanroom methodology? 

Describe the characteristics of a good software reliability model? 
Explain the relevance of security engineering? 

What is a safety-critical system? 


Son EA a ee 


19.8 Summary 


This chapter gave an introduction to some important topics in software engineering 
including software reliability and the Cleanroom methodology, dependability, 
availability, security and safety-critical systems. 

Software reliability is the probability that the program works without failure for 
a period of time, and it is usually expressed as the mean time to failure. Cleanroom 
involves the application of statistical techniques to calculate software reliability, 
and it is based on the expected usage of the software. 

It is essential that the software used in the safety- and security-critical fields is 
dependable, with the software available when required, as well as operating safely 
and reliably without any adverse side effects. Many of these systems are fault 
tolerant and are designed to deal with (and recover) from faults that occur during 
execution. 

These systems need to be secure and able to deal with external attacks, and need 
to include recovery mechanisms to enable normal service to be restored as quickly 
as possible. It is essential that if the system fails then it fails safely. 

Today, billions of computers are connected to the Internet, and this has led to a 
growth in attacks on computers. It is essential that developers are aware of the 
threats facing a system and are familiar with techniques to eliminate them. 
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Formal Methods 20 


20.1 ‘Introduction 


The term “formal methods” refer to various mathematical techniques used for the 
formal specification and development of software. They consist of a formal spec- 
ification language, and employ a collection of tools to support the syntax checking 
of the specification, as well as the proof of properties of the specification. They 
allow questions to be asked about what the system does independently of the 
implementation. 

The use of mathematical notation avoids speculation about the meaning of 
phrases in an imprecisely worded natural language description of a system. Natural 
language is inherently ambiguous, whereas mathematics employs a precise rigorous 
notation. Spivey [Spi:92] defines formal specification as: 


20.1.1 Definition 20.1 (Formal Specification) 


Formal specification is the use of mathematical notation to describe in a precise 
way the properties that an information system must have, without unduly con- 
straining the way in which these properties are achieved. 

The formal specification thus becomes the key reference point for the different 
parties involved in the construction of the system. It may be used as the reference 
point in the requirements; program implementation; testing and program docu- 
mentation. It promotes a common understanding for all those concerned with the 
system. The term “formal methods” is used to describe a formal specification 
language and a method for the design and implementation of computer systems. 
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The specification is written in a mathematical language, and the implementation 
is derived from the specification via step-wise refinement.' The refinement step 
makes the specification more concrete and closer to the actual implementation. 
There is an associated proof obligation to demonstrate that the refinement is valid, 
and that the concrete state preserves the properties of the more abstract state. Thus, 
assuming that the original specification is correct, and the proofs of correctness of 
each refinement step are valid, then there is a very high degree of confidence in the 
correctness of the implemented software. 

Step-wise refinement is illustrated as follows: the initial specification S is the 
initial model Mj; it is then refined into the more concrete model M,, and M, is then 
refined into M>, and so on until the eventual implementation M, = E is produced. 


S = MopCM,CM2CM3C........CM, = E 


Requirements are the foundation of the system to be built, and irrespective of the 
best design and development practices, the product will be incorrect if the 
requirements are incorrect. The objective of requirements validation is to ensure 
that the requirements reflect what is actually required by the customer (in order to 
build the right system). Formal methods may be employed to model the require- 
ments, and the model exploration yields further desirable or undesirable properties. 
The ability to prove that certain properties are true of the specification is very 
valuable, especially in safety critical and security critical applications. These 
properties are logical consequences of the definition of the requirements, and, 
where appropriate, the requirements may need to be amended. Thus, formal 
methods may be employed in a sense to debug the requirements during require- 
ments validation. 

The use of formal methods generally leads to more robust software and to 
increased confidence in its correctness. The challenges involved in the deployment 
of formal methods in an organization include the education of staff in formal 
specification, as the use of these mathematical techniques may be a culture shock to 
many staff. 

Formal methods have been applied to a diverse range of applications, including 
the security critical field; the safety critical field; the railway sector; microprocessor 
verification; the specification of standards, and the specification and verification of 
programs. 

Parnas and others have criticized formal methods on the following grounds 
(Table 20.1): 

However, formal methods are potentially quite useful and reasonably easy to 
use. The use of a formal method such as Z or VDM forces the software engineer to 
be precise and helps to avoid ambiguities present in natural language. Clearly, a 
formal specification should be subject to a peer review to provide confidence in 


‘It is debatable whether step-wise refinement is cost effective in mainstream software engineering, 
as it involves re-writing a specification ad nauseum. It is time-consuming, as significant time is 
required to prove that each refinement step is valid. 
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Table 20.1 Criticisms of formal methods 
No — Criticism 
1 Often the formal specification is as difficult to read as the program” 


Many formal specifications are wrong* 


3) Formal methods are strong on syntax but provide little assistance in deciding on what 
technical information should be recorded using the syntax* 


4 Formal specifications provide a model of the proposed system. However, a precise 
unambiguous mathematical statement of the requirements is what is needed? 


>) Step-wise refinement is unrealistic.° It is like, for example, deriving a bridge from the 
description of a river and the expected traffic on the bridge. There is always a need for a 
creative step in design 


6 Much unnecessary mathematical formalisms have been developed rather than using the 
available classical mathematics’ 


its correctness. New formalisms need to be intuitive to be usable by practitioners. 
The advantage of classical mathematics is that it is familiar to students. 


20.2 Why Should We Use Formal Methods? 


There is a strong motivation to use best practice in software engineering in order to 
produce software adhering to high quality standards. Quality problems with soft- 
ware may cause minor irritations or major damage to a customer’s business 
including loss of life.* Formal methods are a leading-edge technology that may help 
companies to reduce the occurrence of defects in software products. Brown 
[Bro:90] argues that for the safety critical field that: 


?COf course, others might reply by saying that some of Parnas’s tables are not exactly intuitive, and 
that the notation he employs in some of his tables is quite unfriendly. The usability of all of the 
mathematical approaches needs to be enhanced if they are to be taken seriously by industrialists. 
Obviously, the formal specification must be analysed using mathematical reasoning and tools to 
provide confidence in its correctness. The validation may be carried out using mathematical proof 
of key properties of the specification; software inspections; or specification animation. 

4VDM includes a method for software development as well as the specification language. 
>Models are extremely valuable as they allow simplification of the reality. A mathematical study of 
the model demonstrates whether it is a suitable representation of the system. Models allow 
properties of the proposed requirements to be studied prior to implementation. 

’We mentioned the serious problems with the Therac-25 radiotherapy machine in Chap. 18. 
°Step-wise refinement involves rewriting a specification with each refinement step producing a 
more concrete specification (that includes code and formal specification) until eventually the 
detailed code is produced. However, tool support may make refinement easier. 

7Approaches such as VDM or Z are useful in that they add greater rigour to the software 
development process. Classical mathematics is familiar to students and therefore it is desirable that 
new formalisms are introduced only where absolutely necessary. 
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Missile systems must be presumed dangerous until shown to be safe, and that the 
absence of evidence for the existence of dangerous errors does not amount to 
evidence for the absence of danger. 

This suggests that companies in the safety critical field need to demonstrate that 
every reasonable practice was taken to prevent the occurrence of defects. One such 
practice is the use of formal methods, and its exclusion may need to be justified in 
some domains. It is quite possible that a software company may be sued for 
software which injures a third party,’ and this suggests that companies will need a 
rigorous quality assurance system to prevent the occurrence of defects. 

There is some evidence to suggest that the use of formal methods provides 
savings in the cost of the project. For example, a 9% cost saving is attributed to the 
use of formal methods during the CICS project; the T800 project attributes a 
12-month reduction in testing time to the use of formal methods. These are dis- 
cussed in more detail in chapter one of [HB:95]. 

The use of formal methods is mandatory in certain circumstances. The Ministry 
of Defence in the United Kingdom issued two safety-critical standards'° in the 
early 1990s related to the use of formal methods in the software development 
lifecycle. 

The first is Defence Standard 00-55, “The Procurement of safety critical soft- 
ware in defense equipment” [MOD:91a] which makes it mandatory to employ 
formal methods in safety-critical software development in the UK; and mandates 
the use of formal proof that the most crucial programs correctly implement their 
specifications. 

The second is Def Stan 00-56 “Hazard analysis and safety classification of the 
computer and programmable electronic system elements of defense equipment” 
[MOD:9 1b]. The objective of this standard is to provide guidance to identify which 
systems or parts of systems being developed are safety—critical and thereby require 
the use of formal methods. This proposed system is subject to an initial hazard 
analysis to determine whether there are safety-critical parts. 

The reaction to these defence standards 00-55 and 00-56 was quite hostile 
initially, as most suppliers were unlikely to meet the technical and organization 
requirements of the standard [Tie:91]. The standards were subsequently revised to 
be less prescriptive on the use of formal methods. 


°A comprehensive disclaimer of responsibility for problems (rather than a guarantee of quality) 
accompany most software products, and so the legal aspects of licensing software may protect 
software companies from litigation. However, greater legal protection for the customer can be built 
into the contract between the supplier and the customer for bespoke-software development. 
‘The U.K. Defence Standards 0055 and 0056 were later revised to be less prescriptive on the use 
of formal methods. 
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20.3 Applications of Formal Methods 


Formal methods have been employed to verify correctness in the nuclear power 
industry, the aerospace industry, the security technology area, and the railroad 
domain. These sectors are subject to stringent regulatory controls to ensure safety 
and security. Several organizations have piloted formal methods with varying 
degrees of success. These include IBM, who developed VDM at its laboratory in 
Vienna; IBM (Hursley) piloted the Z formal specification language on the CICS 
(Customer Information Control System) project. 

The mathematical techniques developed by Parnas (i.e., tabular expressions) 
have been employed to specify the requirements of the A-7 aircraft as part of a 
research project for the US Navy''. Tabular expressions have also been employed 
for the software inspection of the automated shutdown software of the Darlington 
Nuclear power plant in Canada.'* These are two successful uses of mathematical 
techniques in software engineering. 

There are examples of the use of formal methods in the railway domain, and 
examples dealing with the modeling and verification of a railroad gate controller 
and railway signaling are described in [HB:95]. Clearly, it is essential to verify 
safety critical properties such as “when the train goes through the level crossing 
then the gate is closed”. 


20.4 Tools for Formal Methods 


A key criticism of formal methods is the limited availability of tools to support the 
software engineer in writing a formal specification or in conducting proof. Many of 
the early tools were criticized as not being of industrial strength. However, in recent 
years more advanced tools to support the software engineer’s work in formal 
specification and formal proof have become available, and this should continue in 
the coming years. 

The tools include syntax checkers that determine whether the specification is 
syntactically correct; specialized editors which ensure that the written specification 
is syntactically correct; tools to support refinement; automated code generators that 
generate a high-level language corresponding to the specification; theorem provers 
to demonstrate the presence or absence of key properties and to prove the cor- 
rectness of refinement steps, and to identify and resolve proof obligations; and 
specification animation tools where the execution of the specification can be 
simulated. 


‘However, the resulting software was never actually deployed on the A-7 aircraft. 

This was an impressive use of mathematical techniques and it has been acknowledged that 
formal methods must play an important role in future developments at Darlington. However, given 
the time and cost involved in the software inspection of the shutdown software some managers 
have less enthusiasm in shifting from hardware to software controllers [Ger:94]. 
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The B-Toolkit from B-Core is an integrated set of tools that supports the B- 
Method. These include syntax and type checking, specification animation, proof 
obligation generator, an auto-prover, a proof assistor, and code generation. This 
allows, in theory, a complete formal development from initial specification to final 
implementation to be achieved, with every proof obligation justified, leading to a 
provably correct program. 

The IFAD Toolbox! is a support tool for the VDM-SL specification language, 
and it includes support for syntax and type checking, an interpreter and debugger to 
execute and debug the specification, and a code generator to convert from VDM-SL 
to C++. It also includes support for graphical notations such as the OMT/UML 
design notations. 


20.5 Approaches to Formal Methods 


There are two key approaches to formal methods: namely the model-oriented 
approach of VDM or Z, and the algebraic or axiomatic approach of the process 
calculi such as the calculus communicating systems (CCS) or communicating 
sequential processes (CSP). 


20.5.1 Model-Oriented Approach 


The model-oriented approach to specification is based on mathematical models, and 
a model is a mathematical representation or abstraction of a physical entity or 
system. The model aims to provide a mathematical explanation of the behaviour of 
the physical world, and it is considered suitable if its properties closely match those 
of the system being modeled. A model will allow predictions of future behaviour to 
be made, and many models are employed in the physical world (e.g., weather 
forecasting system). 

It is fundamental to explore the model to determine its adequacy, and to 
determine the extent to which it explains the underlying physical behaviour, and 
allows predictions of future behaviour to be made. This will determine its accept- 
ability as a representation of the physical world. Models that are ineffective will be 
replaced with models that offer a better explanation of the manifested physical 
behaviour. There are many examples in science of the replacement of one theory by 
a newer one. For example, the Copernican model of the universe replaced the older 
Ptolemaic model, and Newtonian physics was replaced by Einstein’s theories on 
relativity [Kuh:70]. 

The model-oriented approach to software development involves defining an 
abstract model of the proposed software system. The model acts as a representation 


13The IFAD Toolbox has been renamed to VDMTools as IFAD sold the VDM Tools to CSK in 
Japan. 
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of the proposed system, and the model is then explored to assess its suitability. The 
exploration of the model takes the form of model interrogation, i.e., asking ques- 
tions and determining the effectiveness of the model in answering the questions. 
The modeling in formal methods is typically performed via elementary discrete 
mathematics, including set theory, sequences, functions and relations. 

VDM and Z are model-oriented approaches to formal methods. VDM arose from 
work done in the IBM laboratory in Vienna in formalizing the semantics for the 
PL/1 compiler, and it was later applied to the specification of software systems. The 
origin of the Z specification language is in work done at Oxford University in the 
early 1980s. 


20.5.2 Axiomatic Approach 


The axiomatic approach focuses on the properties that the proposed system is to 
satisfy, and there is no intention to produce an abstract model of the system. The 
required properties and behaviour of the system are stated in mathematical notation. 
The difference between the axiomatic specification and a model-based approach is 
may be seen in the example of a stack. 

The stack includes operators for pushing an element onto the stack and popping 
an element from the stack. The properties of pop and push are explicitly defined in 
the axiomatic approach. The model-oriented approach constructs an explicit model 
of the stack and the operations are defined in terms of the effect that they have on 
the model. The specification of the pop operation on a stack is given by axiomatic 
properties, for example, pop (push(s,x)) = s. 


20.5.3 Comment 20.2 (Axiomatic Approach) 


The property-oriented approach has the advantage that the implementer is not 
constrained to a particular choice of implementation, and the only constraint is that 
the implementation must satisfy the stipulated properties. 

The emphasis is on the required properties of the system, and implementation 
issues are avoided. The focus is on the specification of the underlying behaviour, 
and properties are typically stated using mathematical logic or higher-order logics. 
Mechanized theorem-proving techniques may be employed to prove results. 

One potential problem with the axiomatic approach is that the properties spec- 
ified may not be satisfiable in any implementation. Thus, whenever a “formal 
axiomatic theory” is developed a corresponding “model” of the theory must be 
identified, in order to ensure that the properties may be realized in practice. That is, 
when proposing a system that is to satisfy some set of properties, there is a need to 
prove that there is at least one system that will satisfy the set of properties. 
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20.6 Proof and Formal Methods 


A mathematical proof typically includes natural language and mathematical sym- 
bols, and often many of the tedious details of the proof are omitted. The proof of a 
conjecture may be by a “divide and conquer” technique; i.e., breaking the con- 
jecture down into subgoals and then attempting to prove the subgoals. Many proofs 
in formal methods are concerned with crosschecking the details of the specification, 
or checking the validity of refinement steps, or checking that certain properties are 
satisfied by the specification. There are often many tedious lemmas to be proved, 
and theorem provers ’“ are essential in assisting with this. Machine proof needs to be 
explicit, and reliance on some brilliant insight is avoided. Proofs by hand are 
notorious for containing errors or jumps in reasoning, while machine proofs are 
explicit but are often extremely lengthy and unreadable (e.g., the actual machine 
proof of correctness of the VIPER microprocessor’> [Tie:91] consisted of several 
million formulae). 

A formal mathematical proof consists of a sequence of formulae, where each 
element is either an axiom or derived from a previous element in the series by 
applying a fixed set of mechanical rules. 

Theorem provers are invaluable in resolving many of the thousands of proof 
obligations that arise from a formal specification, and it is not feasible to apply 
formal methods in an industrial environment without the use of machine-assisted 
proof. Automated theorem proving is difficult, as often mathematicians prove a 
theorem with an initial intuitive feeling that the theorem is true. Human intervention 
to provide guidance or intuition improves the effectiveness of the theorem prover. 

The proof of various properties about a program increases confidence in its 
correctness. However, an absolute proof of correctness'° is unlikely except for the 
most trivial of programs. A program may consist of legacy software that is assumed 
to work; a compiler that is assumed to work correctly creates it. Theorem provers 
are programs that are assumed to function correctly. The best that formal methods 
can claim is increased confidence in correctness of the software, rather than an 
absolute proof of correctness. 


4Many existing theorem provers are difficult to use and are for specialist use only. There is a need 
to improve their usability. 

'SThis verification was controversial with RSRE and Charter overselling VIPER as a chip design 
that conforms to its formal specification. 

‘This position is controversial with others arguing that if correctness is defined mathematically 
then the mathematical definition (i.e., formal specification) is a theorem, and the task is to prove 
that the program satisfies the theorem. They argue that the proofs for non-trivial programs exist, 
and that the reason why there are not many examples of such proofs is due to a lack of 
mathematical specifications. 
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20.7. The Future of Formal Methods 


The debate concerning the level of use of mathematics in software engineering is 
still ongoing. Most practitioners are against the use of mathematics and avoid its 
use. They tend to employ methodologies such as software inspections and testing to 
improve confidence in the correctness of the software. Industrialists often need to 
balance conflicting needs such as quality; cost; and aggressive time pressures. They 
argue that commercial realities dictate that appropriate methodologies and tech- 
niques are required that allow them to achieve their business goals in a timely 
manner. 

The other camp argues that the use of mathematics is essential in the delivery of 
high-quality and reliable software, and that if a company does not place sufficient 
emphasis on quality it will pay the price in terms of a poor reputation in the market 
place. 

It is generally accepted that mathematics and formal methods must play a role in 
the safety critical and security critical fields. Apart from that the extent of the use of 
mathematics is a hotly disputed topic. The pace of change in the world is 
extraordinary, and companies face major competitive pressures in a global market 
place. It is unrealistic to expect companies to deploy formal methods unless they 
have clear evidence that it will support them in delivering commercial products to 
the market place ahead of their competition, at the right price and with the right 
quality. Formal methods needs to prove that it can do this if it wishes to be taken 
seriously in mainstream software engineering. The issue of technology transfer of 
formal methods to industry is discussed in [ORg: 17a]. 


20.8 The Vienna Development Method 


VDM dates from work done by the IBM research laboratory in Vienna. This group 
was specifying the semantics of the PL/1 programming language using an opera- 
tional semantic approach (discussed in Chap. 12). That is, the semantics of the 
language were defined in terms of a hypothetical machine, which interprets the 
programs of that language [BjJ:78, BjJ:82]. Later work led to the Vienna Devel- 
opment Method (VDM) with its specification language, Meta IV. This was used to 
give the denotational semantics of programming languages; i.e., a mathematical 
object (set, function, etc.) is associated with each phrase of the language [BjJ:82]. 
The mathematical object is termed the denotation of the phrase. 

VDM is a model-oriented approach and this means that an explicit model of the 
state of an abstract machine is given, and operations are defined in terms of this 
state. Operations may act on the system state, taking inputs, and producing outputs 
as well as a new system state. Operations are defined in a precondition and 
post-condition style. Each operation has an associated proof obligation to ensure 
that if the precondition is true, then the operation preserves the system invariant. 
The initial state itself is, of course, required to satisfy the system invariant. 
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VDM uses keywords to distinguish different parts of the specification, e.g., 
preconditions, post-conditions, as introduced by the keywords pre and post 
respectively. In keeping with the philosophy that formal methods specifies what a 
system does as distinct from how, VDM employs post-conditions to stipulate the 
effect of the operation on the state. The previous state is then distinguished by 
employing hooked variables, e.g., v~, and the postcondition specifies the new state 
which is defined by a logical predicate relating the pre-state to the post-state. 

VDM is more than its specification language VDM-SL, and is, in fact, a software 
development method, with rules to verify the steps of development. The rules 
enable the executable specification, i.e., the detailed code, to be obtained from the 
initial specification via refinement steps. Thus, we have a sequence S = So, Sj, ..., Sp 
= E of specifications, where S is the initial specification, and E is the final (exe- 
cutable) specification. 

Retrieval functions enable a return from a more concrete specification to the 
more abstract specification. The initial specification consists of an initial state, a 
system state, and a set of operations. The system state is a particular domain, where 
a domain is built out of primitive domains such as the set of natural numbers, etc., 
or constructed from primitive domains using domain constructors such as Cartesian 
product, disjoint union, etc. A domain-invariant predicate may further constrain the 
domain, and a type in VDM reflects a domain obtained in this way. Thus, a type in 
VDM is more specific than the signature of the type, and thus represents values in 
the domain defined by the signature, which satisfy the domain invariant. In view of 
this approach to types, it is clear that VDM types may not be “statically type 
checked”. 

VDM specifications are structured into modules, with a module containing the 
module name, parameters, types, operations, etc. Partial functions occur frequently 
in computer science as many functions, may be undefined, or fail to terminate for 
some arguments in their domain. VDM addresses partial functions by employing 
nonstandard logical operators, namely the logic of partial functions (LPFs), which 
was discussed in Chap. 16. 

VDM has been used in industrial projects, and its tool support includes the IFAD 
Toolbox.!’ There are several variants of VDM, including VDM**, the 
object-oriented extension of VDM, and the Irish school of the VDM, which is 
discussed in the next section. 


20.9 VDM*, the Irish School of VDM 


The Irish School of VDM is a variant of standard VDM, and is characterized by 
[Mac:90] its constructive approach, classical mathematical style, and its terse 
notation. This method aims to combine the what and how of formal methods in that 
its terse specification style stipulates in concise form what the system should do; 


’The VDM Tools are now available from the CSK Group in Japan. 
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furthermore, the fact that its specifications are constructive (or functional) means 
that the how is included with the what. However, it is important to qualify this by 
stating that the how as presented by VDM* is not directly executable, as several of 
its mathematical data types have no corresponding structure in high-level pro- 
gramming languages or functional languages. Thus, a conversion or reification of 
the specification into a functional or higher-level language must take place to ensure 
a successful execution. Further, the fact that a specification is constructive is no 
guarantee that it is a good implementation strategy, if the construction itself is 
naive. 

The Irish school follows a similar development methodology as in standard 
VDM, and is a model-oriented approach. The initial specification is presented, with 
initial state and operations defined. The operations are presented with precondi- 
tions; however, no postcondition is necessary as the operation is “functionally” (i.e., 
explicitly) constructed. 

There are proof obligations to demonstrate that the operations preserve the 
invariant. That is, if the precondition for the operation is true, and the operation is 
performed, then the system invariant remains true after the operation. The philos- 
ophy is to exhibit existence constructively rather than a theoretical proof of exis- 
tence that demonstrates the existence of a solution without presenting an algorithm 
to construct the solution. 

The school avoids the existential quantifier of predicate calculus and reliance on 
logic in proof is kept to a minimum, and emphasis instead is placed on equational 
reasoning. Structures with nice algebraic properties are sought, and one nice 
algebraic structure employed is the monoid, which has closure, associativity, and a 
unit element. The concept of isomorphism is powerful, reflecting that two structures 
are essentially identical, and thus we may choose to work with either, depending on 
which is more convenient for the task in hand. 

The school has been influenced by the work of Polya and Lakatos. The former 
[Pol:57] advocated a style of problem solving characterized by first considering an 
easier sub-problem, and considering several examples. This generally leads to a 
clearer insight into solving the main problem. Lakatos’s approach to mathematical 
discovery [Lak:76] is characterized by heuristic methods. A primitive conjecture is 
proposed and if global counter-examples to the statement of the conjecture are 
discovered, then the corresponding hidden lemma for which this global coun- 
terexample is a local counter example is identified and added to the statement of the 
primitive conjecture. The process repeats, until no more global counterexamples are 
found. A skeptical view of absolute truth or certainty is inherent in this. 

Partial functions are the norm in VDM*, and as in standard VDM, the problem 
is that functions may be undefined, or fail to terminate for several of the arguments 
in their domain. The logic of partial functions (LPFs) is avoided, and instead care is 
taken with recursive definitions to ensure termination is achieved for each argu- 
ment. Academic and industrial projects have been conducted using the method of 
the Irish school, but at this stage tool support is limited. 
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20.10 The Z Specification Language 


Z is a formal specification language founded on Zermelo set theory, and it was 
developed by Abrial at Oxford University in the early 1980s. It is a model-oriented 
approach where an explicit model of the state of an abstract machine is given, and 
the operations are defined in terms of the effect on the state. It includes a mathe- 
matical notation that is similar to VDM, and it employs the visually striking schema 
calculus, which consists essentially of boxes, with these boxes or schemas used to 
describe operations and states. The schema calculus enables schemas to be used as 
building blocks and combined with other schemas. The Z specification language 
was published as an ISO standard (ISO/IEC 13568:2002) in 2002. 

The schema calculus is a powerful means of decomposing a specification into 
smaller pieces or schemas. This helps to make Z specification highly readable, as 
each individual schema is small in size and self-contained. The exception handling 
is done by defining schemas for the exception cases, and these are then combined 
with the original operation schema. Mathematical data types are used to model the 
data in a system and these data types obey mathematical laws. These laws enable 
simplification of expressions and are useful with proofs. 

Operations are defined in a precondition/postcondition style. However, the 
precondition is implicitly defined within the operation; i.e., it is not separated out as 
in standard VDM. Each operation has an associated proof obligation to ensure that 
if the precondition is true, then the operation preserves the system invariant. The 
initial state itself is, of course, required to satisfy the system invariant. Postcondi- 
tions employ a logical predicate which relates the pre-state to the post-state, and the 
post-state of a variable v is given by priming, e.g., v’.. Various conventions are 
employed, e.g., v? indicates that v is an input variable and v/ indicates that v is an 
output variable. The symbol = Op operation indicates that this operation does not 
affect the state, whereas A Op indicates that this operation that affects the state. 

Many data types employed in Z have no counterpart in standard programming 
languages. It is therefore important to identify and describe the concrete data 
structures that will ultimately represent the abstract mathematical structures. The 
operations on the abstract data structures may need to be refined to yield operations 
on the concrete data structure that yield equivalent results. For simple systems, 
direct refinement (i.e., one step from abstract specification to implementation) may 
be possible; in more complex systems, deferred refinement is employed, where a 
sequence of increasingly concrete specifications are produced to yield the exe- 
cutable specification eventually. 

Z has been successfully applied in industry, and one of its well-known successes 
is the CICS project at IBM Hursley in England. Z is described in more detail in 
Chap. 21. 
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20.11 The B Method 


The B-Technologies [McD:94] consist of three components: a method for software 
development, namely the B-Method; a supporting set of tools, namely, the B- 
Toolkit; and a generic program for symbol manipulation, namely, the B-Tool (from 
which the B-Toolkit is derived). The B-Method is a model-oriented approach and is 
closely related to the Z specification language. Abrial developed the B specification 
language, and every construct in the language has a set theoretic counterpart, and 
the method is founded on Zermelo set theory. Each operation has an explicit 
precondition. 

One key purpose [McD:94] of the abstract machine in the B-Method is to 
provide encapsulation of variables representing the state of the machine and 
operations that manipulate the state. Machines may refer to other machines, and a 
machine may be introduced as a refinement of another machine. The abstract 
machines are specification machines, refinement machines, or implementable 
machines. The B-Method adopts a layered approach to design where the design is 
gradually made more concrete by a sequence of design layers. Each design layer is 
a refinement that involves a more detailed implementation in terms of abstract 
machines of the previous layer. The design refinement ends when the final layer is 
implemented purely in terms of library machines. Any refinement of a machine by 
another has associated proof obligations, and proof is required to verify the validity 
of the refinement step. 

Specification animation of the Abstract Machine Notation (AMN) specification 
is possible with the B-Toolkit, and this enables typical usage scenarios of the AMN 
specification to be explored for requirements validation. This is, in effect, an early 
form of testing, and it may be used to demonstrate the presence or absence of 
desirable or undesirable behavior. Verification takes the form of a proof to 
demonstrate that the invariant is preserved when the operation is executed within its 
precondition, and this is performed on the AMN specification with the B-Toolkit. 

The B-Toolkit provides several tools that support the B-Method, and these 
include syntax and type checking; specification animation, proof obligation gen- 
erator, auto prover, proof assistor, and code generation. Thus, in theory, a complete 
formal development from initial specification to final implementation may be 
achieved, with every proof obligation justified, leading to a provably correct 
program. 

The B-Method and toolkit have been successfully applied in industrial appli- 
cations, including the CICS project at IBM Hursley in the United Kingdom 
[Hor:95]. The automated support provided has been cited as a major benefit of the 
application of the B-Method and the B-Toolkit. 
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20.12 Predicate Transformers and Weakest Preconditions 


The precondition of a program S is a predicate, i.e., a statement that may be true or 
false, and it is usually required to prove that if the precondition Q is true,; i.e., {Q} 
S {R}, then execution of S is guaranteed to terminate in a finite amount of time in a 
state satisfying R. 

The weakest precondition of a command S with respect to a postcondition 
R represents the set of all states such that if execution begins in any one of these 
states, then execution will terminate in a finite amount of time in a state with R true 
[Gri:81]. These set of states may be represented by a predicate Q’, so that wp(S,R) = 
wps (R) = Q’, and so wps is a predicate transformer, i.e., it may be regarded as a 
function on predicates. The weakest precondition is the precondition that places the 
fewest constraints on the state than all of the other preconditions of (S,R). That is, 
all of the other preconditions are stronger than the weakest precondition. 

The notation Q{S}R is used to denote partial correctness and indicates that if 
execution of S commences in any state satisfying Q, and if execution terminates, 
then the final state will satisfy R. Often, a predicate Q which is stronger than the 
weakest precondition wp(S,R) is employed, especially where the calculation of the 
weakest precondition is nontrivial. Thus, a stronger predicate Q such that Q > wp 
(S,R) is sometimes employed. 

There are many properties associated with the weakest preconditions, and these 
may be used to simplify expressions involving weakest preconditions, and in 
determining the weakest preconditions of various program commands such as 
assignments, iterations, etc. Weakest preconditions may be used in developing a 
proof of correctness of a program in parallel with its development [Dij:76]. 

An imperative program may be regarded as a predicate transformer. This is since 
a predicate P characterizes the set of states in which the predicate P is true, and an 
imperative program may be regarded as a binary relation on states, which may be 
extended to a function F, leading to the Hoare triple P{F}Q. That is, the program 
F acts as a predicate transformer with the predicate P regarded as an input assertion, 
i.e., a Boolean expression that must be true before the program F is executed, and 
the predicate Q is the output assertion, which is true if the program F terminates 
(where F commenced in a state satisfying P). 


20.13. The Process Calculi 


The objectives of the process calculi [Hor:85] are to provide mathematical models 
that provide insight into the diverse issues involved in the specification, design, and 
implementation of computer systems which continuously act and interact with their 
environment. These systems may be decomposed into sub-systems that interact 
with each other and their environment. 
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The basic building block is the process, which is a mathematical abstraction of 
the interactions between a system and its environment. A process that lasts indef- 
initely may be specified recursively. Processes may be assembled into systems; they 
may execute concurrently; or communicate with each other. Process communica- 
tion may be synchronized, and this takes the form of a process outputting a message 
simultaneously to another process inputting a message. Resources may be shared 
among several processes. Process calculi such as CSP [Hor:85] and CCS [Mil:89] 
have been developed to enrich the understanding of communication and concur- 
rency, and these calculi obey a rich collection of mathematical laws. 

The expression (a ? P) in CSP describes a process which first engages in event a, 
and then behaves as process P. A recursive definition is written as (1X)*F(X), and 
the example of a simple chocolate vending machine is given recursively as: 


VMS = uX: {coin, choc} e (coin ? (choc ? X)) 


The simple vending machine has an alphabet of two symbols, namely, coin and 
choc. The behaviour of the machine is that a coin is entered into the machine, and 
then a chocolate selected and provided. 

CSP processes use channels to communicate values with their environment, and 
input on channel c is denoted by (c?.x P,). This describes a process that accepts any 
value x on channel c, and then behaves as process P,. In contrast, (c!e P) defines a 
process which outputs the expression e on channel c and then behaves as process P. 

The m-calculus is a process calculus based on names. Communication between 
processes takes place between known channels, and the name of a channel may be 
passed over a channel. There is no distinction between channel names and data 
values in the m-calculus. The output of a value v on channel a is given by day; ie., 
output is a negative prefix. Input on a channel a is given by a(x), and is a positive 
prefix. Private links or restrictions are given by (x)P in the m-calculus. 


20.14 The Parnas Way 


Parnas has been influential in the computing field, and his ideas on the specification, 
design, implementation, maintenance, and documentation of computer software 
remain important. He advocates a solid classical engineering approach to devel- 
oping software, and he argues that the role of an engineer is to apply scientific 
principles and mathematics in designing and developing software products. His 
main contributions to software engineering are summarized in Table 20.2 below: 
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Table 20.2 Parnas’s contributions to software engineering 


Area Description 


Tabular expressions These are mathematical tables for specifying requirements, and enable 
complex predicate logic expressions to be represented in a simpler form 


Mathematical He advocates the use of precise mathematical documentation 

documentation 

Requirements He advocates the use of mathematical relations to specify the requirements 
specification precisely 

Software design He developed information hiding which is used in object-oriented design’® , 


and allows software to be designed for change [Par:72]. Every 
information-hiding module has an interface that provides the only means to 
access the services provided by the modules. The interface hides the module’s 
implementation 


Software inspections His approach requires the reviewers to take an active part in the inspection. 
They are provided with a list of questions by the author and their analysis 
involves the production of mathematical table to justify the answers 


Predicate logic He developed an extension of the predicate calculus to deal with partial 
functions. This approach preserves the classical two-valued logic and deals 
with undefined values that may occur in predicate logic expressions 


20.15 Usability of Formal Methods 


There are practical difficulties associated with the use of formal methods. It seems to 
be assumed that programmers and customers are willing to become familiar with 
the mathematics used in formal methods. There is little evidence to suggest that 
customers in mainstream organizations would be prepared to use formal methods’. 
Customers are concerned with their own domain and speak the technical language 
of that domain*”. Often, the use of mathematics is an alien activity that bears little 
resemblance to their normal work. Programmers are interested in programming 
rather than in mathematics, and generally have no interest in becoming 
mathematicians”'. 


'8It is surprising that many in the object-oriented world seem unaware that information hiding goes 
back to the early 1970s and many have never heard of Parnas. 

'The domain in which the software is being used will influence the willingness or otherwise of the 
customers to become familiar with the mathematics required. There is very little interest from 
customers in mainstream software engineering, and the perception is that formal methods are 
difficult to use. However, in some domains such as the regulated sector there is a greater 
willingness of customers to become familiar with the mathematical notation. 

0The author’s experience is that most customers have a very limited interest in using mathematics. 
?!Mathematics that is potentially useful to software engineers was discussed in Chap. 18. 
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Table 20.3 Techniques for validation of formal specification 


Technique Description 


Proof This involves demonstrating that the formal specification adheres to key 
properties of the requirements. The implementation will need to preserve 
these properties also 


Software This involves a Fagan like inspection to perform the validation. It may 
inspections involve comparing an informal set of requirements (unless the customer 

has learned the formal method) with the formal specification 
Specification This involves program (or specification) execution as a way to validate 
animation the formal specification. It is similar to testing 


However, the mathematics involved in most formal methods is reasonably ele- 
mentary, and, in theory, if both customers and programmers are willing to learn the 
formal mathematical notation, then a rigorous validation of the formal specification 
can take place to verify its correctness. Both parties can review the formal speci- 
fication to verify its correctness, and the code can be verified to be correct with 
respect to the formal specification. It is usually possible to get a developer to learn a 
formal method, as a programmer has some experience of mathematics and logic; 
however, in practice, it is more difficult to get a customer to learn a formal method. 

This means that often a formal specification of the requirements and an informal 
definition of the requirements using a natural language are maintained. It is essential 
that both of these documents are consistent and that there is a rigorous validation of 
the formal specification. Otherwise, if the programmer proves the correctness of the 
code with respect to the formal specification, and the formal specification is 
incorrect, then the formal development of the software is incorrect. There are 
several techniques to validate a formal specification (Table 20.3) and these are 
described in [Wic:00]: 


20.15.1 Why are Formal Methods difficult? 


Formal methods are perceived as being difficult to use and of offering limited value 
in mainstream software engineering. Programmers receive some training in math- 
ematics as part of their education. However, in practice, most programmers who 
learn formal methods at university never use formal methods again once they take 
an industrial position. 

It may well be that the very nature of formal methods is such that it is suited only 
for specialists with a strong background in mathematics. Some of the reasons why 
formal methods are perceived as being difficult are (Table 20.4): 
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Table 20.4 Factors in Difficulty of Formal Methods 


Factor Description 

Notation/Intuition The notation employed differs from that used in mathematics. Many 
programmers find the notation in formal methods to be unintuitive 

Formal specification It is easier to read a formal specification than to write one 

Validation of formal The validation of a formal specification using proof techniques or a 

specification Fagan like inspection is difficult 

Refinement” The refinement of a formal specification into successive more 


concrete specifications with proof of validity of each refinement 
step is difficult and time consuming 


Proof Proof can be difficult and time consuming 


Tool Support Many of the existing tools are difficult to use 


20.15.2 Characteristics of a Usable Formal Method 


It is important to investigate ways by which formal methods can be made more 
usable to software engineers. This may involve designing more usable notations 
and better tools to support the process. Practical training and coaching to employees 
can help also. Some of the characteristics of a usable formal method are 
(Table 20.5): 


Table 20.5 Characteristics of a usable formal method 


Characteristic Description 

Intuitive A formal method should be intuitive 

Teachable A formal method needs to be teachable to the average software engineer. 
The training should include (at least) writing practical formal 
specifications 

Tool support Good tools to support formal specification, validation, refinement and 
proof are required 

Adaptable to Change is common in a software engineering environment. A usable 

change formal method should be adaptable to change 

Technology The process for software development needs to be defined to include 

transfer path formal methods. The migration to formal methods needs to be managed 

Cost” The use of formal methods should be cost effective with a return on 


investment. There should be benefits in time, quality and productivity 


It is highly unlikely that refinement is cost effective for mainstream software engineering. 
However, it may be useful in the regulated environment. 

3 commercial company will expect a return on investment from the use of a new technology. 
This may be reduced software development costs, improved quality, improved timeliness of 
projects or improvements in productivity. 
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20.16 Review Questions 


1. What are formal methods and describe their potential benefits? How essential is tool support? 
2. What is stepwise refinement and is it realistic in mainstream software engineering? 
3. Discuss Parnas’s criticisms of formal methods and discuss whether his views are justified 


4. Discuss the applications of formal methods and which areas have benefited most from their 
use? What problems have arisen? 


5. Describe a technology transfer path for the potential deployment of formal methods in an 
organization 


6. Explain the difference between the model-oriented approach and the axiomatic approach 
7. Discuss the nature of proof in formal methods and tools to support proof 


8. Discuss the Vienna Development Method and explain the difference between standard VDM 
and VDM* 


9. Discuss Z and B? Describe the tools in the B-Toolkit 


10. Discuss process calculi such as CSP, CCS or z-calculus 


20.17. Summary 


This chapter discussed formal methods, which are a rigorous approach to the 
development of high-quality software. Formal methods employ mathematical 
techniques for the specification and formal development of software, and are very 
useful in the safety critical field. They consist of formal specification languages or 
notations; a methodology for formal software development; and a set of tools to 
support the syntax checking of the specification, as well as the proof of properties of 
the specification. 

Formal methods allow questions to be asked and answered about what the 
system does independently of the implementation. The use of formal methods 
generally leads to more robust software and to increased confidence in its cor- 
rectness. There are challenges involved in the deployment of formal methods, as the 
use of these mathematical techniques may be a culture shock to many staff. 

Formal methods may be model oriented or axiomatic oriented. The 
model-oriented approach includes formal methods such as VDM, Z and B. The 
axiomatic approach includes the process calculi such as CSP, CCS and the z calculus. 

The usability of formal methods was considered as well as an examination of 
why formal methods are difficult and what the characteristics of a usable formal 
method would be. 


® 
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21.1. Introduction 


Z is a formal specification language based on Zermelo set theory. It was developed 
at the Programming Research Group at Oxford University in the early 1980s [1] and 
became an ISO standard in 2002. Z specifications are mathematical and employ a 
classical two-valued logic. The use of mathematics ensures precision and allows 
inconsistencies and gaps in the specification to be identified. Theorem provers may 
be employed to demonstrate that the software implementation meets its 
specification. 

Z is a ‘model-oriented’ approach with an explicit model of the state of an 
abstract machine given, and operations are defined in terms of this state. Its 
mathematical notation is used for formal specification, and the schema calculus is 
used to structure the specifications. The schema calculus is visually striking and 
consists essentially of boxes, and these boxes or schemas are used to describe 
operations and states. The schemas may be used as building blocks and combined 
with other schemas. The simple schema below (Fig. 21.1) is the specification of the 
positive square root of a real number. 

Schema calculus is a powerful means of decomposing a specification into 
smaller pieces or schemas. This helps to make Z specifications highly readable, as 
each individual schema is small in size and self-contained. Exception handling is 
addressed by defining schemas for the exception cases. These are then combined 
with the original operation schema. Mathematical data types are used to model the 
data in a system, and these data types obey mathematical laws. These laws enable 
simplification of expressions and are useful with proofs. 
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- SqgRoot———— 
num?, root! : R 


num? >0 
root!’ = num? 


root! >0 


Fig. 21.1 Specification of positive square root 


Operations are defined in a precondition/postcondition style. A precondition 
must be true before the operation is executed, and the postcondition must be true 
after the operation has been executed. The precondition is implicitly defined within 
the operation. Each operation has an associated proof obligation to ensure that if the 
precondition is true, then the operation preserves the system invariant. The system 
invariant is a property of the system that must be true at all times. The initial state 
itself is, of course, required to satisfy the system invariant. 

The precondition for the specification of the square root function above is that 
num? > O, i.e. the function SgRoot may be applied to positive real numbers only. 
The postcondition for the square root function is root!? = num? and root! > 0. 
That is, the square root of a number is positive and its square gives the number. 
Postconditions employ a logical predicate that relates the pre-state to the post-state, 
with the post-state of a variable being distinguished by priming the variable, e.g. v’. 

Z is a typed language and whenever a variable is introduced, its type must be 
given. A type is simply a collection of objects, and there are several standard types 
in Z. These include the natural numbers N, the integers Z and the real numbers R. 
The declaration of a variable x of type X is written x: X. It is also possible to create 
your own types in Z. 

Various conventions are employed within Z specification, for example, v? 
indicates that v is an input variable; v! indicates that v is an output variable. The 
variable num? is an input variable and roof! is an output variable for the square root 
example above. The notation & in a schema indicates that the operation Op does not 
affect the state; whereas the notation A in the schema indicates that Op is an 
operation that affects the state. 

Many of the data types employed in Z have no counterpart in standard pro- 
gramming languages. It is, therefore, important to identify and describe the concrete 
data structures that ultimately will represent the abstract mathematical structures. As 
the concrete structures may differ from the abstract, the operations on the abstract 
data structures may need to be refined to yield operations on the concrete data that 
yield equivalent results. For simple systems, direct refinement (i.e. one step from 
abstract specification to implementation) may be possible; in more complex 
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systems, deferred refinement! is employed, where a sequence of increasingly 
concrete specifications is produced to yield the executable specification. There is a 
calculus for combining schemas to make larger specifications, and this is discussed 
later in the chapter. 


Example 21.1 The following is a Z specification to borrow a book from a library 
system. The library is made up of books that are on the shelf, books that are 
borrowed and books that are missing (Fig. 21.2). The specification models a library 
with sets representing books on the shelf, on loan or missing. These are three 
mutually disjoint subsets of the set of books Bkd-Id. 


The system state is defined in the Library schema below, and operations such as 
Borrow and Return affect the state. The Borrow operation is specified below 
(Fig. 21.3). 

The notation PBkd-Id is used to represent the power set of Bkd-Id (i.e. the set of 
all subsets of Bkd-Id). The disjointness condition for the library is expressed by the 
requirement that the pair-wise intersection of the subsets on-shelf, borrowed and 
missing is the empty set. 

The precondition for the Borrow operation is that the book must be available on 
the shelf to borrow. The postcondition is that the borrowed book is added to the set 
of borrowed books and is removed from the books on the shelf. 

Z has been successfully applied in the industry including the CICS project at 
IBM Hursley in the UK.’ Next, we describe key parts of Z including sets, relations, 
functions, sequences and bags. 


21.2 Sets 


Sets were discussed in Chap. 2 and this section focuses on their use in Z. Sets may 
be enumerated by listing all of their elements. Thus, the set of all even natural 
numbers less than or equal to 10 is 


{2,4, 6,8, 10}. 


Sets may be created from other sets using set comprehension, i.e. stating the 
properties that its members must satisfy. For example, the set of even natural 
numbers less than 10 is given by set comprehension as 


{n: N|n 40 An<10A n mod 2=0-n}. 


'Step-wise refinement involves producing a sequence of increasingly more concrete specifications 
until eventually the executable code is produced. Each refinement step has associated proof 
obligations to prove that the refinement step is valid. 

°This project claimed a 9% increase in productivity attributed to the use of formal methods. 
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- Library 
on-shelf, missing, borrowed : P_ Bkd-Id 


on-shelf ~ missing = © 
on-shelf (\ borrowed = © 
borrowed ~\ missing = © 


Fig. 21.2 Specification of a library system 


— Borrow 
A Libra 
b? :Bkd-Id 


‘Be € on-shelf 
on-shelf’ = on-shelf \ {b?} 
borrowed’ = borrowed U {b?} 


Fig. 21.3 Specification of Borrow operation 


There are three main parts to the set comprehension above. The first part is the 
signature of the set and this is given by n: N above. The first part is separated from 
the second part by a vertical line. The second part is given by a predicate, and for 
this example, the predicate is n 40 An<10An mod 2 = 0. The second part is 
separated from the third part by a bullet. The third part is a term, and for this 
example, it is simply n. The term is often a more complex expression, e.g. log(n’). 

In mathematics, there is just one empty set. However, since Z is a typed set 
theory, there is an empty set for each type of set. Hence, there are an infinite number 
of empty sets in Z. The empty set is written as [X] where X is the type of the empty 
set. However, X is generally omitted when the type is clear. 

Various operations on sets such as union, intersection, set difference and sym- 
metric difference are employed in Z. The power set of a set X is the set of all subsets 
of X and is denoted by P X. The set of non-empty subsets of X is denoted by P|X 
where 


P\X == {U :PX|U4O[x}}. 
A finite set of elements of type X (denoted by F X) is a subset of X that cannot 


be put into a one-to-one correspondence with a proper subset of itself. This is 
defined formally as 


FX == {U : PX |-dV:PU-V4UA (af: V > > U)}. 
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The expression f : V > — U denotes that f is a bijection from U to V and 
injective, surjective and bijective functions were discussed in Chap. 2. 

The fact that Z is a typed language means that whenever a variable is introduced 
(e.g. in quantification with V and 3), it is first declared. For example, Vj: J.-P = Q. 
There is also the unique existential quantifier 4\j : J | P which states that there is 
exactly one j of type J that has property P. 


21.3 Relations 


Relations are used extensively in Z and were discussed in Chap. 2. A relation R 
between X and Y is any subset of the Cartesian product of X and Y, i.e. 
RC(X x Y), and a relation in Z is denoted by R : X < Y. The notation x+y 
indicates that the pair (vy) € R. 

Consider the relation home_owner: Person «+ Home that exists between people 
and their homes. An entry daphne '> mandalay € home_owner if daphne is the 
owner of mandalay. It is possible for a person to own more than one home: 


rebecca'> nirvana € home _owner, 
rebecca> tivoli © home_owner. 
It is possible for two people to share ownership of a home: 
rebecca} nirvana € home _owner, 
lawrence +> nirvana € home_owner. 


There may be some people who do not own a home, and there is no entry for 
these people in the relation home_owner. The type Person includes every possible 
person, and the type Home includes every possible home. The domain of the 
relation home_owner is given by 


x € dom home_owner <= dh : Home - xt>h € home _owner. 


The range of the relation home_owner is given by 


h € ranhome_owner <> Ax : Person: xt>h € home _owner. 


The composition of two relations home_owner: Person ~ Home and home_- 
value: Home «> Value yields the relation owner_wealth: Person — Value and is 
given by the relational composition home_owner; home_value where 
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Pv € home_owner; home_value = 


(Sh : Home - p++ h € home_owner \ ht v € home_value). 
The relational composition may also be expressed as 
owner_wealth = home_value 0 home_owner. 


The union of two relations often arises in practice. Suppose a new entry aisling 
+> muckross is to be added. Then this is given by 


home_owner' = home_owner U {aisling > muckross}. 


Suppose that we are interested in knowing all females who are house owners. 
Then we restrict the relation home_owner so that the first element of all ordered 
pairs has to be female. Consider female: P Person with {aisling, rebecca} C 
female. 


home _owner = {aisling — muckross, rebecca‘ nirvana, 


lawrence + nirvana}, 


female < home_owner = {aisling > muckross, rebecca nirvana}. 


That is, female <1 home_owner is a relation that is a subset of home_owner, and 
the first element of each ordered pair in the relation is female. The operation <I is 
termed domain restriction and its fundamental property is 


xeyeEeUdIRS (xXEUAxHyER}, 


where R : XY and U: P X. 
There is also a domain anti-restriction (subtraction) operation and its funda- 
mental property is 


xHeye€ U--PRS(x€UAxH yeER}, 
where R : X + Y and U: PX. 


There are also range restriction (the <loperator) and the range anti-restriction 
operator (the —< <loperator). These are discussed in [1]. 
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21.4 Functions 


A function [1] is an association between objects of some type X and objects of 
another type Y such that given an object of type X, there exists only one object in 
Y associated with that object. A function is a set of ordered pairs where the first 
element of the ordered pair has at most one element associated with it. A function 
is, therefore, a special type of relation, and a function may be total or partial. 

A total function has exactly one element in Y associated with each element of X, 
whereas a partial function has at most one element of Y associated with each 
element of X (there may be elements of X that have no element of Y associated with 
them). 

A partial function from X to Y(f : XY) is a relation f : X < Y such that: 


Vx: Xsy,z:Y-(xeyefAxezefsy=z). 


The association between x and y is denoted by f(x) = y, and this indicates that the 
value of the partial function f at x is y. A total function from X to Y (denoted 
f : X — Y)isa partial function such that every element in X is associated with some 
value of Y. 


[:X —~Yof:X+YAdom f =X. 


Clearly, every total function is a partial function but not vice versa. 
One operation that arises quite frequently in specifications is the function 
override operation. Consider the following specification of a temperature map: 


— TempMap 
CityList : PCity 


temp : City #Z 


dom temp = CityList 


Suppose the temperature map is given by 
temp = {Cork+> 17, Dublin 19, London 15}. Then consider the problem of 
updating the temperature map if a new temperature reading is made in Cork, e.g. 
{Cork 18}. Then the new temperature chart is obtained from the old temperature 
chart by function override to yield {Cork+> 18, Dublin 19, London 15}. This 
is written as 


temp’ = temp © {Cork > 18}. 
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The function override operation combines two functions of the same type to give 
a new function of the same type. The effect of the override operation is that the 
entry {Cork 17} is removed from the temperature chart and replaced with the 
entry {Cork 18}. 

Suppose f, g : X-Yare partial functions, then f @ g is defined and indicates that 
f is overridden by g. It is defined as follows: 


(f ® g)(x) = g(x), where x € dom g. 
(f ® g)(x) =f(x), where x ¢ dom g Ax € domf. 


This may also be expressed (using domain anti-restriction) as 


f ® g = ((domg)—— pf) Ug. 


There is a notation in Z for injective, surjective and bijective functions. An 
injective function is one to one, i.e. 


A surjective function is onto, 1.e. 
Given y € Y, dx € X such that f(x) = y. 


A bijective function is one to one and onto, and it indicates that the sets X and 
Y can be put into one-to-one correspondence with one another. Z includes lambda 
calculus notation (A-calculus was discussed in Chap. 12) to define functions. For 
example, the function cube == Ax: N-x * x* x. Function composition fi g is 
similar to relational composition. 


21.5 Sequences 


The type of all sequences of elements drawn from a set X is denoted by seq 
X. Sequences are written as (x1, x2, .... x,) and the empty sequence is denoted by ). 
Sequences may be used to specify the changing state of a variable over time, with 
each element of the sequence representing the value of the variable at a 
discrete-time instance. 

Sequences are functions and a sequence of elements drawn from a set X is a 
finite function from the set of natural numbers to X. The domain of the function 
consists of all numbers between | and # f (where #f is the cardinality of f). It is 
defined formally as 


seqX == {f : N-+X|domf = 1... #f -f}. 
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The sequence (x1, x2, .... X,) above is given by 
{lr x, 2%, .. 2. Ne Xp}. 


There are various functions to manipulate sequences. These include the sequence 
concatenation operation. Suppose o = (x1, x2, ....X,) and T= (y1, yo, ----¥m), 
then 


o _ T= (x1, X25 ++ ++Xny V1, 2; -- erie 
The head of a non-empty sequence gives the first element of the sequence. 
head o = head (x,, x2, ...-Xn) = 1. 


The tail of a non-empty sequence is the same sequence except that the first 
element of the sequence is removed. 


tail o = tail(x1, x2, ....Xn) = (2, .-.-Xn). 


Suppose f : X — Y and a sequence o: seq X, then the function map applies f to 
each element of o: 


mapfo = mapf (x1, x2, ..-.%n) = (f (x1), f (x2), ----f (Xn). 
The map function may also be expressed via function composition as 
mapfo = o;f. 
The reverse order of a sequence is given by the rev function: 


rev O = rev(x1, X2, ----Xn) = (Xn, + X2, X1). 


21.6 Bags 


A bag is similar to a set except that there may be multiple occurrences of each 
element in the bag. A bag of elements of type X is defined as a partial function from 
the type of the elements of the bag to positive whole numbers. The definition of a 
bag of type X is 


bag X == X+N). 
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For example, a bag of marbles may contain three blue marbles, two red marbles 
and one green marble. This is denoted by B = [1 ,b,b,g,,7,r]. The bag of marbles is, 
thus, denoted by 


bag Marble == Marble+N,. 


The function count determines the number of occurrences of an element in a bag. 
For the example above, count Marble b = 3 and count Marble y = 0 since there are 
no yellow marbles in the bag. This is defined formally as 


count bag X y = 0 y ¢ bag X. 
count bag X y = (bagX)(y) y € bag X. 
An element y is in bag X if and only if y is in the domain of bag X. 
yin bagX = y € dom (bag X). 


The union of two bags of marbles B, = [I D,b,),g,,7,r] and B2 = [I b,g,,7,y] is 
given by B,| WB» = [b,b,b,b, g,g,r,r,r,y]. It is defined formally as 


(B\ WB>) (y) = B2(y) y ¢ dom Br Ay € dom Bz 
(B,YB2)(y)= Bi(y) ye€domB, Ay ¢dom Bo 
(B\YB2)(y)= Bi(y) y€dom B, Ay ¢dom Bo. 


A bag may be used to record the number of occurrences of each product in a 
warehouse as part of an inventory system. It may model the number of items 
remaining for each product in a vending machine (Fig. 21.4). 

The operation of a vending machine would require other operations such as 
identifying the set of acceptable coins, checking that the customer has entered 
sufficient coins to cover the cost of the good, returning change to the customer and 
updating the quantity on hand of each good after a purchase. A more detailed 
examination is in [1]. 


-A Vending Machine 
stock : bag Good 
price : Good >N, 


dom stock € dom price 


Fig. 21.4 Specification of Vending Machine using Bags 
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21.7. Schemas and Schema Composition 


The schemas in Z are visually striking and the specification is presented in 
two-dimensional graphic boxes. Schemas are used for specifying states and state 
transitions, and they employ notation to represent the before and after state (e.g. 
s ands’ where s’ represents the after state of s). They group all relevant information 
that belongs to a state description. 

There are a number of useful schema operations such as schema inclusion, 
schema composition and the use of propositional connectives to link schemas 
together. The A convention indicates that the operation affects the state whereas the 
= convention indicates that the state is not affected. These operations and con- 
ventions allow complex operations to be specified concisely and assist with the 
readability of the specification. Schema composition is analogous to relational 
composition and allows new schemas to be derived from existing schemas. 

A schema name S, may be included in the declaration part of another schema Sp. 
The effect of the inclusion is that the declarations in S; are now part of S2 and the 
predicates of S; are S, are joined together by conjunction. If the same variable is 
defined in both S; and S5, then it must be of the same type in both schemas. 


S, | 3 
xy: N| Si;z:N 
| 


xty>2| Z=xt+y 


The result is that Sjincludes the declarations and predicates of S; (Fig. 21.5). 

Two schemas may be linked by propositional connectives such as S; A So, 
S; V So, Sj = Sz and S; = Sj. The schema S; V Sz is formed by merging the 
declaration parts of S; and Sz and then combining their predicates by the logical V 
operator. For example, S = S; V Sp yields (Fig. 21.6). 

Schema inclusion and the linking of schemas use normalization to convert 
sub-types to maximal types, and predicates are employed to restrict the maximal 
type to the sub-type. This involves replacing declarations of variables (e.g. u: 1.0.35 
with u: Z, and adding the predicate u > 0 and u < 36 to the predicate part of the 
schema). 

The A and & conventions are used extensively, and the notation A TempMap is 
used in the specification of schemas that involve a change of state. The notation A 
TempMap represents 


A TempMap = TempMap \ TempMap'. 
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Fig. 21.5 Schema inclusion 


> 
xy:N 
ziNX 


KP po2lav 2=x+y 


Fig. 21.6 Merging schemas (S; V S2) 


The longer form of A TempMap is written as follows: 


— ATempMap. 


CityList, CityList’ : P City 


temp, temp’ : City + Z 


dom temp = CityList 


dom temp’ = CityList’ 


The notation & TempMap is used in the specification of operations that do not 
involve a change to the state. 


| — & TempMap. 


| ATempMap 


| CityList = CityList’ 


| temp = temp’ 


21.7. Schemas and Schema Composition 


Table. 21.1. Schema composition 


Step Procedure 

1 Rename all after state variables in S to something new: 
Salsa sie 

2 Rename all before state variables in T to the same new thing, i.e. 
T [s* /s]. 

3 Form the conjunction of the two new schemas, 
S [st /s'] AT [s* /s]. 

4 Hide the variable introduced in steps 1 and 2 


Se Tt=(S [er /s AT [s* fs) \(™) 
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Schema composition is analogous to relational composition and it allows new 
specifications to be built from existing ones. It allows the after state variables of one 
schema to be related with the before variables of another schema. The composition 
of two schemas S and T (S; T) is described in detail in [1] and involves four steps 


(Table 21.1). 


The example below should make schema composition clearer. Consider the 


composition of S and T where S and T are defined as follows: 


s eee 
x,x’,y? :N| xx’ tN 
| 
x'=y?-2| x’=x+1 
| 
8; | "rj 
xxty? s N | xx’ N 
| 
xt=y?-2| xve=xt +] 


S, and T, represent the results of step 1 and step 2, with x’ renamed to x* in S, 


and x renamed to x* in T. Step 3 and step 4 yield (Fig. 21.7). 


Schema composition is useful as it allows new specifications to be created from 


existing ones. 


-S, AT -S ;T— 

x,x",x',y? oN x, x’, yP oN 

x =y?-2 dx*:Ne 

x =x +1 (x* =y?-2 
x’=x' +1) 


Fig. 21.7 Schema composition 
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21.8 Reification and Decomposition 


A Z specification involves defining the state of the system and then specifying the 
required operations. The Z specification language employs many constructs that are 
not part of conventional programming languages, and a Z specification is, therefore, 
not directly executable on a computer. A programmer implements the formal 
specification, and mathematical proof may be employed to prove that a program 
meets its specification. 

Often, there is a need to write an intermediate specification that is between the 
original Z specification and the eventual program code. This intermediate specifi- 
cation is more algorithmic and uses less abstract data types than the Z specification. 
The intermediate specification is termed the design and the design needs to be 
correct with respect to the specification, and the program needs to be correct with 
respect to the design. The design is a refinement (reification) of the state of the 
specification, and the operations of the specification have been decomposed into 
those of the design. 

The representation of an abstract data type such as a set by a sequence is termed 
data reification, and data reification is concerned with the process of transforming 
an abstract data type into a concrete data type. The abstract and concrete data types 
are related by the retrieve function, and the retrieve function maps the concrete data 
type to the abstract data type. There are typically several possible concrete data 
types for a particular abstract data type (i.e. refinement is a relation), whereas there 
is one abstract data type for a concrete data type (i.e. retrieval is a function). For 
example, sets are often reified to unique sequences; however, more than one unique 
sequence can represent a set whereas a unique sequence represents exactly one set. 

The operations defined on the concrete data type are related to the operations 
defined on the abstract data type. That is, the commuting diagram property is 
required to hold (Fig. 21.8). That is, for an operation EJ on the concrete data type to 
correctly model the operation © on the abstract data type, the diagram must 
commute, and the commuting diagram property requires proof. That is, it is 
required to prove that 


ret(oLIt) = (ret o) © (rett). 


retr(o) © retr(t) 


retr(o), 


retr 
retr(t) 


(ot) 


(o O17) 


Fig. 21.8 Refinement commuting diagram 
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In Z, the refinement and decomposition are done with schemas. It is required to 
prove that the concrete schema is a valid refinement of the abstract schema, and this 
gives rise to a number of proof obligations. It needs to be proved that the initial 
states correspond to one another and that each operation in the concrete schema is 
correct with respect to the operation in the abstract schema and also that it is 
applicable (i.e. whenever the abstract operation may be performed, the concrete 
operation may also be performed). 


21.9 Proof in Z 


Mathematicians perform rigorous proof of theorems using technical and natural 
language. Logicians employ formal proofs to prove theorems using propositional 
and predicate calculus. Formal proofs generally involve a long chain of reasoning 
with every step of the proof justified. Rigorous proofs involve precise reasoning 
using a mixture of natural and mathematical language. Rigorous proofs [1] have 
been described as being analogous to high-level programming languages, whereas 
formal proofs are analogous to machine language. 

A mathematical proof includes natural language and mathematical symbols, and 
often many of the tedious details of the proof are omitted. Many proofs in formal 
methods such as Z are concerned with crosschecking on the details of the speci- 
fication or on the validity of the refinement step or proofs that certain properties are 
satisfied by the specification. There are often many tedious lemmas to be proved, 
and tool support is essential as proof by hand often contains errors or jumps in 
reasoning. Machine proofs are lengthy and largely unreadable; however, they 
provide extra confidence as every step in the proof is justified. 

The proof of various properties about the programs increases confidence in their 
correctness. 


21.10 Review Questions 


1. Describe the main features of the Z specification language. 

2. Explain the difference between P, X, P X and FX. 

3. Give an example of a set derived from another set using set compre- 
hension. Explain the three main parts of set comprehension in Z. 

4. Discuss the applications of Z and which areas have benefited most from 
their use. What problems have arisen? 

5. Give examples to illustrate the use of domain and range restriction 
operators and domain and range anti-restriction operators with relations 
in Z. 

6. Give examples to illustrate relational composition. 
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7. Explain the difference between a partial and total function, and give 
examples to illustrate function override. 
8. Give examples to illustrate the various operations on sequences including 
concatenation, head, tail, map and reverse operations. 
9. Give examples to illustrate the various operations on bags. 
10. Discuss the nature of proof in Z and tools to support proof. 
11. Explain the process of refining an abstract schema to a more concrete 
representation, the proof obligations that are generated and the com- 
muting diagram property. 


21.11 Summary 


Z is a formal specification language that was developed in the early 1980s at Oxford 
University in England. It has been employed in both industry and academia, and it 
was used successfully on IBM’s CICS project. Its specifications are mathematical, 
and this leads to more rigorous software development. Its mathematical approach 
allows properties to be proved about the specification, and any gaps or inconsis- 
tencies in the specification may be identified. 

Z is a ‘model-oriented’ approach, and an explicit model of the state of an abstract 
machine is given, and the operations are defined in terms of their effect on the state. 
Its main features include a mathematical notation that is similar to VDM, and the 
schema calculus consists essentially of boxes that are used to describe operations 
and states. 

The schema calculus enables schemas to be used as building blocks to form 
larger specifications. It is a powerful means of decomposing a specification into 
smaller pieces and helps with the readability of Z specifications, as each individual 
schema is small in size and self-contained. 

Z is a highly expressive specification language, and it includes notation for sets, 
functions, relations, bags, sequences, predicate calculus and schema calculus. 
Z specifications are not directly executable as many of its data types and constructs 
are not part of modern programming languages. Therefore, there is a need to refine 
the Z specification into a more concrete representation and prove that the refinement 
is valid. 
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22.1 ‘Introduction 


Statistics is an empirical science that is concerned with the collection, organization, 
analysis, interpretation and presentation of data. The data collection needs to be 
planned and this may include surveys and experiments. Statistics are widely used 
by government and industrial organizations, and they are employed for forecasting 
as well as for presenting trends. They allow the behaviour of a population to be 
studied and inferences to be made about the population. These inferences may be 
tested (hypothesis testing) to ensure their validity. 

The analysis of statistical data allows an organization to understand its perfor- 
mance in key areas and to identify problematic areas. Organizations will often 
examine performance trends over time and will devise appropriate plans and actions 
to address problematic areas. The effectiveness of the actions taken will be judged 
by improvements in performance trends over time. 

It is often not possible to study the entire population, and instead, a represen- 
tative subset or sample of the population is chosen. This random sample is used to 
make inferences regarding the entire population, and it is essential that the sample 
chosen is indeed random and representative of the entire population. Otherwise, the 
inferences made regarding the entire population will be invalid, as a selection bias 
has occurred. Clearly, a census where every member of the population is sampled is 
not subject to this type of bias. 

A statistical experiment is a causality study that aims to draw a conclusion 
between the values of a predictor variable(s) and a response variable(s). For 
example, a statistical experiment in the medical field may be conducted to 
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determine if there is a causal relationship between the use of a particular drug and 
the treatment of a medical condition such as lowering cholesterol in the population. 
A Statistical experiment involves the following: 


— Planning the research. 
Designing the experiment. 
— Performing the experiment. 
Analysing the results. 

— Presenting the results. 


22.2 Basic Statistics 


The field of statistics is concerned with summarizing, digesting and extracting 
information from large quantities of data. It provides a collection of methods for 
planning an experiment and analysing data to draw accurate conclusions from the 
experiment. We distinguish between descriptive statistics and inferential statistics: 
Descriptive Statistics 

This is concerned with describing the information in a set of data elements in 
graphical format or describing its distribution. 

Inferential Statistics 

This is concerned with making inferences with respect to the population by using 
information gathered in the sample. 


22.2.1 Abuse of Statistics 


Statistics are extremely useful in drawing conclusions about a population. However, 
it is essential that the random sample chosen is actually random, and that the 
experiment is properly conducted to ensure that valid conclusions are inferred. 
Some examples of the abuse of statistics include the following: 


— The sample size may be too small to draw conclusions. 

— It may not be a genuine random sample of the population. 

— There may be bias introduced from poorly worded questions. 
— Graphs may be drawn to exaggerate small differences. 

— Area may be misused in representing proportions. 

— Misleading percentages may be used. 
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The quantitative data used in statistics may be discrete or continuous. Discrete 
data is numerical data that has a finite or countable number of possible values, and 
continuous data is numerical data that has an infinite number of possible values. 


22.2.2 Statistical Sampling and Data Collection 


Statistical sampling is concerned with the methodology for choosing a random 
sample of a population and the study of the sample with the goal of drawing valid 
conclusions about the entire population. If a genuine representative random sample 
of the population is chosen, then a detailed study of the sample will provide insight 
into the whole population. This helps to avoid a lengthy expensive (and potentially 
infeasible) study of the entire population. 

The sample chosen must be truly random and the sample size sufficiently large to 
enable valid conclusions to be drawn for the entire population. The probability of 
being chosen for the random sample should be the same for each member of the 
population. 

Random Sample 
A random sample is a sample of the population such that each member of the 
population has an equal chance of being chosen. 

A large sample size gives more precise information about the population. 
However, little extra information is gained from increasing the sample size above a 
certain level, and the sample size chosen will depend on factors such as money and 
time available, the aims of the survey, the degree of precision required and the 
number of subsamples required. Table 22.1 summarizes several ways of generating 
a random sample from the population: 

Once the sample is chosen, the next step is to obtain the required information 
from the sample. The data collection may be done by interviewing each member in 
the sample, conducting a telephone interview with each member, conducting a 
postal questionnaire survey and so on (Table 22.2). 

The design of the questionnaire requires careful consideration as a poorly 
designed questionnaire may lead to invalid results. The questionnaire should be as 
short as possible, and the questions should be simple and unambiguous. Closed 
questions where the respondent chooses from simple categories are useful. It is best 
to pilot the questionnaire prior to carrying out the survey. 


22.3 Frequency Distribution and Charts 


The data gathered from a statistical study is often raw and may yield little infor- 
mation as it stands. Therefore, the way the data is presented is important, and it is 
useful to present the information in pictorial form. The advantage of a pictorial 
presentation is that it allows the data to be presented in an attractive and colourful 
way, and the reader is not overwhelmed with excessive detail. This enables analysis 
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Table 22.1 Sampling techniques 


Sampling 
technique 


Systematic 
sampling 
Stratified 


sampling 


Multi-Stage 


sampling 


Cluster sampling 


Convenience 
sampling 


Description 


The population is listed and every kth member of the population is 
sampled. For example, to choose a 2% (1 in 50) sample, every 50th 
member of the population would be sampled 


The population is divided into two or more strata and each subpopulation 
(stratum) is then sampled. Each element in the subpopulation shares the 
same characteristics (e.g. age groups, gender). The results from the various 
strata are then combined 


This approach may be used when the population is spread over a wide 
geographical area. The area is split up into a number of regions, and a 
small number of the regions are randomly selected. Each selected region is 
then sampled. It requires less effort and time but it may introduce bias if a 
small number of regions are selected, as it is not a truly random sample 


A population is divided into clusters and a few of these clusters are 
exhaustively sampled (i.e. every element in the cluster is considered). This 
approach may lead to significant selection bias, as the sampling is not 
random 


Sampling is done as convenient, and in this case, each person selected 
may decide whether to participate or not in the sample 


Table 22.2 Types of survey 


Survey type 


Personal interview 


Phone survey 


Mail questionaire 
survey 


Direct 
measurement 


Direct 
observational study 


Experiment 


Description 


Interviews are expensive and time consuming, but allow detailed and 
accurate information to be collected. Questionnaires are often employed 
and the interviewers need to be trained in interview techniques. 
Interviews need to be planned and scheduled, and they are useful in 
dealing with issues that may arise (e.g. a respondent not fully 
understanding a question) 


This is a reasonably efficient and cost-effective way to gather data. 
However, refusals or hang-ups may affect the outcome. It also has an 
in-built bias as only those people with telephones may be contacted and 
interviewed 


This involves sending postal questionnaire survey to the participants. 
The questionnaire needs to be well designed to ensure the respondents 
understand the questions and answer them correctly. There is a danger 
of a low response rate that may invalidate the findings 


This may involve direct measurement of all those in the sample (e.g. the 
height of all students in a class) 

An observational study allows individuals to be studied and the 
variables of interest to be measured 


An experiment imposes some treatment on individuals in order to study 
the response 
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and conclusions to be drawn. There are several types of charts or graphs that are 
often employed in the presentation of the data including the following: 


— Bar chart. 

— Histogram. 
Pie Chart. 
Trend Graph. 


| 


| 


A frequency table is often used to present and summarize data, where a simple 
frequency distribution consists of a set of data values and the number of items that 
have that value (i.e. a set of data values and their frequency). The information is 
then presented pictorially in a bar chart. 

The general frequency distribution is employed when dealing with a larger 
number of data values (e.g. >20 data values). It involves dividing the data into a set 
of data classes and listing the data classes in one column and the frequency of data 
values in that category in another column. The information is then presented pic- 
torially in a bar chart or histogram. 

Figure 22.1 presents the raw data of salaries earned by different people in a 
company, and Table 22.3 presents the raw data in table format using a frequency 
table of salaries. Figure 22.2 presents a bar chart of the salary data in pictorial 
form, and it is much easier to read than the raw data presented in Fig. 22.1. 

A histogram is a way of representing data in bar chart format, and it shows the 
frequency or relative frequency of various data values or ranges of data values. It is 
usually employed when there are a large number of data values, and it gives a crisp 
picture of the spread of the data values and the centring and variance of the data 
values from the mean. 

The data is divided into disjoint intervals where an interval is a certain range of 
values. The horizontal axis of the histogram contains the intervals (also known as 
buckets) and the vertical axis shows the frequency (or relative frequency) of each 
interval. 


Fig. 22.1 Raw salary data 


Table 22.3 Frequency table Salary Frequency 
of salary data 

45,000 2 

50,000 8 

65,000 at 

90,000 1 
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Frequency Distribution - Salary 


= 


Frequency 


Frequency 
ONH DOO 


45k 50k 65k 90k 
Salary 


Fig. 22.2 Bar chart of salary data 


The bars represent the frequency and there is no space between the bars. The 
histogram has an associated shape, e.g. it may be a normal distribution, a bimodal 
or multi-modal distribution, and it may be positively or negatively skewed. The 
variation and centring refer to the spread of data, and the spread of the data is 
important as it may indicate whether the entity under study (e.g. a process) is too 
variable or whether it is performing within the requirements. 

The histogram is termed process-centred if its centre coincides with the customer 
requirements; otherwise, the process is too high or too low. A histogram allows 
predictions of future performance to be made, where it can be assumed that the 
future will resemble the past. 

The construction of a histogram first requires that a frequency table be con- 
structed, and this requires that the range of the data values be determined. The data 
are divided into a number of classes (or data buckets), where a bucket is a particular 
range of data values, and the relative frequency of each bucket is displayed in bar 
format. The number of class intervals or buckets is determined, and the class 
intervals are defined. The class intervals are mutually disjoint and span the range of 
the data values. Each data value belongs to exactly one class interval, and the 
frequency of each class interval is determined (Table 22.4). 

The results of a class test in mathematics are summarized in Table 24.4. There 
are 30 students in the class and each student achieves a score somewhere between 0 
and 100. There are four data intervals between 0 and 100 employed to summarize 
the scores, and the result of each student belongs to exactly one interval. 
Figure 22.3 is the associated histogram for the frequency data, and it gives a 
pictorial representation of the marks for the class test. 


Table 22.4 Frequency table — yar Frequency 
—test results 

0-24 3 

25-49 10 

50-74 15 


75-100 » 
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Histogram for Class Marks 


Greduency, Frequency 
0 
0-24 25-49 50-74 75-100 
Marks 
Fig. 22.3 Histogram class test results 
Frequency 
T% 10% 
m0-24 
25-49 
33% w50 - 74 
50% 75-100 


Fig. 22.4 Pie chart test results 


We may also employ a pie chart as an alternate way to present the class marks. 
The frequency table is constructed, and a visual representation of the percentage in 
each data class (i.e. the relative frequency) is provided with the pie chart. Each 
portion of the pie chart represents the percentage of the data values in that interval 
(Fig. 22.4). 

We present the monthly sales and profit figures for a company in Table 22.5, and 
Fig. 22.5 gives a pictorial representation of the data in the form of a time series (or 
trend chart). 


22.4 Statistical Measures 


Statistical measures are concerned with the basic analysis of the data to determine 
the average of the data as well as how spread out the data is. The term ‘average’ 
generally refers to the arithmetic mean of a sample, but it may also refer to the 
statistical mode or median of the sample. We first discuss the arithmetic mean as it 
is the mathematical average of the data and is representative of the data. The 
arithmetic mean is the most widely used average in statistics. 
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Table 22.5 Monthly Sales and Profit 


Sales Profit 
Jan 5500 200 
Feb 3000 400 
Mar 3500 200 
Apr 3000 600 
May 4500 100 
Jun 6200 1200 
Jul 7350 3200 
Aug 4100 100 
Sep 9000 3300 
Oct 2000 500 
Nov 1100 800 
Dec 3000 300 


Annual Sales / Profit 
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Fig. 22.5 Monthly sales and profit 


22.4.1. Arithmetic Mean 


The arithmetic mean (or just mean) of a set of n numbers is defined to be the sum of 
the data values divided by the number of values. That is, the arithmetic mean of a 


set of data values x1, x2, .... X, (where the sample size is n) is given by 
n 
xi 
= i=l 
i= 
n 


The arithmetic mean is representative of the data as all values are used in its 
calculation. The mean of the set of values 5, 11, 9, 4, 16, 9 is given by 


m=5+114+9+4+16+9/6 = 54/6 =9. 
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The formula for the arithmetic mean of a set of data values given by a frequency 
table needs to be adjusted. 


xX, x2 age Gio Xn 


fi h bp 


x 2 5 7 10 12 
fe 2 4 7 4 2 


The mean is given by 


m= (2*24+5*447*7+10*4+4 12 *2)/(2+44+74+4+42) 
= (44 20+49 +40 + 24) /19 = 137/19 = 7.2. 


The actual mean of the population is denoted by p, and it may differ from the 
sample mean m. 


22.4.2 Mode 


The mode is the most popular element in the sample, i.e. it is the data element that 
occurs most frequently in the sample. For example, consider a shop that sells 
mobile phones, then the mode of the annual sales of phones is the most popular 
phone sold. The mode of the list [1, 4, 1, 2, 7, 4, 3, 2, 4] is 4, whereas there is no 
unique mode in the sample [1, 1, 3, 3, 4], and it is said to be bimodal. 

The mode of the following frequency distribution is 7 since it occurs the most 
times in the sample. 


x a 5 il 10 12 
@ 2) 4 7 4 2 


It is possible that the mode is not unique (i.e. there are at least two elements that 
occur with the equal highest frequency in the sample), and if this is the case, then 


372 22 Statistics 


we are dealing with a bimodal or possibly a multi-modal distribution (where there 
are more than two elements that occur most frequently in the sample). 


22.4.3 Median 


The median of a set of data is the value of the data item that is exactly half way 
along the set of items, where the data set is arranged in increasing order of 
magnitude. 

If there are an odd number of elements in the sample, the median is the middle 
element. Otherwise, the median is the arithmetic mean of the two middle elements. 

The median of 34, 21, 38, 11, 74, 90, 7 is determined by first ordering the set as 
7, 11, 21, 34, 38, 74, 90, and the median is then given by the value of the fourth 
item in the list which is 34. 

The median of the list 2, 4, 8, 12, 20, 30 is the mean of the middle two items (as 
there are an even number of elements and the set is ordered), and so it is given by 
the arithmetic mean of the third and fourth elements, i.e. ed tli = 10. 

The calculation of the median of a frequency distribution first requires the cal- 
culation of the total number of data elements (i.e. this is given by N = Xf) and then 
determining the value of the middle element in the table, which is the “*'/2 element. 

The median for the following frequency distribution is calculated by 


x 2 5 7 10 12 
Se 2 4 7 4 Dp 


The number of elements is given by N = Lf = 2+4+7+4+42 = 19 and so 
the middle element is given by the value of the “*'/2 element, i.e. the 19+17, = the 
tenth element. From an examination of the table, it is clear that the value of the tenth 
element is 7 and so the median of the frequency distribution is 7. 

The final average that we consider is the midrange of the data in the sample, and 
this is given by the arithmetic mean of the highest and lowest data elements in the 
sample. That is, #miq = Qmax + Xmin) / 2. 

The mean, mode and median coincide for symmetric frequency distributions but 
differ for left- or right-skewed distributions (Fig. 22.6). Skewness describes how 
non-symmetric the data is. 

Dispersion indicates how spread out or scattered the data is, and there are several 
ways of measuring dispersion including how skewed the distribution is, the range of 
the data, variance and the standard deviation. 
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Mode 
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Fig. 22.6 Symmetric distribution 


22.5 Variance and Standard Deviation 


An important characteristic of a sample is its distribution and the spread of each 
element from some measure of central tendency (e.g. the mean). One elementary 
measure of dispersion is that of the sample range, which is defined to be the 
difference between the maximum and minimum value in the sample. That is, the 
sample range is defined to be 


range = Xmax — Xmin- 


The sample range is not a reliable measure of dispersion as just two elements in 
the sample are used, and so extreme values in the sample may distort the range and 
make it very large even if most of the elements are quite close to one another. 

The standard deviation is the most common way to measure dispersion, and it 
gives the average distance of each element in the sample from the arithmetic mean. 
The sample standard deviation of a sample x1, x2, ... X, is denoted by s, and its 
calculation first requires the calculation of the sample mean. It is defined by 


7 — | xt — ni 
- n-1 — 
The population standard deviation is denoted by o and is defined by 


oa fleece) _ foto 


Variance is another measure of dispersion and it is defined as the square of the 
standard deviation. The sample variance s° is given by 
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n—1l n—-1l 


The population variance 6? is given by 


2 Siwy? _ Dae ae 


oO = 
n n 


Example (Standard Deviation) 
Calculate the standard deviation of the samples 2, 4, 6, 8. 
Solution (Standard Deviation) 
The sample mean is given by m=2+4+6+4+8/42=5. 
The sample variance is given by 


s = (2-45) + (4-5) 46-5) + (8-5) 4-1 
=9+1+4+14+9/3 


The sample standard deviation is given by the square root of the variance and so 
it is given by 


s = 6.66 
= 2.58. 


The formula for the standard deviation and variance may be adjusted for fre- 
quency distributions. The standard deviation and mean often go hand in hand, and 
for normal distributions, 68% of the data lies within one standard deviation of the 
mean; 95% of the data lies within two standard deviations of the mean and the vast 
majority (99.7%) of the data lies within three standard deviations of the mean. All 
data values are used in the calculation of the mean and standard deviation, and so 
these measures are truly representative of the data. 


22.6 Correlation and Regression 


The two most common techniques for exploring the relationship between two 
variables are correlation and linear regression. Correlation is concerned with 
quantifying the strength of the relationship between two variables by measuring the 
degree of ‘scatter’ of the data values, whereas regression expresses the relationship 
between the variable in the form of an equation (usually a linear equation). 
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Correlation quantifies the strength and direction of the relationship between two 
numeric variables X and Y, and the correlation coefficient may be positive or 
negative and it lies between —1 and + 1. If the correlation is positive, then as the 
value of one variable increases, the value of the other variable increases (i.e. the 
variables move together in the same direction), whereas if the correlation is neg- 
ative, then as the value of one variable increases ,the value of the other variable 
decreases (i.e. the variables move together in the opposite directions). The corre- 
lation coefficient r is given by the formula: 


XY —XY Xi¥; — OXY; 
Corr(X, Y) = = ia BES Sha PEP TEL 


~ Std(X)Std(Y) yndx _ (Ox) Vad eS y,)° : 


The sign of the correlation coefficient indicates the direction of the relationship 
between the two variables. A correlation of r= + 1 indicates perfect positive 
correlation, whereas a correlation of r = —1 indicates perfect negative correlation. 
A correlation close to zero indicates no relationship between the two variables; a 
correlation of r = —0.3 indicates a weak negative relationship; whereas a correlation 
of r = 0.85 indicates a strong positive relationship. The extent of the relationship 
between the two variables may be seen from the following: 


A change in the value of X leads to a change in the value of Y. 
— A change in the value of Y leads to a change in the value of X. 
Changes in another variable lead to changes in both X and Y. 
— There is no relationship (or correlation) between X and Y. 


The relationship (if any) between the two variables can be seen by plotting the 
values of X and Y in a scatter graph as in Fig. 22.7 and Fig. 22.8. The correlation 
coefficient identifies linear relationships between X and Y, but it does not detect 
non-linear relationships. It is possible for correlation to exist between two variables 
but for no causal relationship to exist, i.e. correlation is not the same as causation. 
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Fig. 22.7 Strong positive correlation 
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Example (Correlation) 

The data in Table 22.6 is a summary of the cost of maintenance of eight printers, 
and it is wished to explore the extent to which the age of the machine is related to 
the cost of maintenance. It is required to calculate the correlation coefficient. 

Solution (Correlation) 

For this example, n = 8 (as there are 8 printers) and ©X;, YY;, UX; Yj, Dp. LY? 
are computed in the last row of the table and so 


xX; = 76; 

LY; = 940; 
LX;Y; = 12680; 
EX? = 978; 
LY? = 169450. 


We input these values into the correlation formula and get 


8 « 12680 — 76 « 940 
— 
V8 « 978 — 762v/8 * 169450 — 9402 
30000 
/2048,/472000 
_ 30000 
~~ 45.25 * 687.02 
30000 
~ 31087 
= 0.96. 


Therefore, r = 0.96 and so there is a strong correlation between the age of the 
machine and the cost of maintenance of the machine. 
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Fig. 22.8 Strong negative correlation 
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Table 22.6 Cost of X (Age) Y (Cost) XY x2 y2 
maintenance of printers 5 50 250 25 2500 
12 1315) 1620 144 18,225 
4 60 240 16 3600 
20 300 6000 400 90,000 
2 25 50 4 625 
10 80 800 100 6400 
15 200 3000 225 40,000 
8 90 720 64 8100 
76 940 12,680 978 169,450 


22.6.1 Regression 


Regression is used to study the relationship (if any) between dependent and inde- 
pendent variables and to predict the dependent variable when the independent 
variable is known. The prediction capability of regression makes it a more powerful 
tool than correlation, and regression is useful in identifying which factors impact 
the desired outcome variable. 

There are several types of regression that may be employed such as linear or 
polynomial regression, and this section is concerned with linear regression where 
the relationship between the dependent and independent variables is expressed by a 
straight line. More advanced statistical analysis may be conducted with multiple 
regression models, where there are several independent variables that are believed 
to affect the value of another variable. 

Regression analysis first involves data gathering and plotting the data on a 
scatter graph. The regression line is the line that best fits the data on the scatter 
graph (Fig. 22.9), and it is usually determined using the method of least squares or 
one of the methods summarized in Table 22.7. The regression line is a plot of the 
expected values of the dependant variable for all values of the independent variable, 
and the formula (or equation) of the regression line is of the form y = mx + b, 
where the coefficients of a and b are determined. 

The regression line then acts as a model that describes the relationship between 
the two variables, and the value of the dependent variable may be predicted from 
the value of the independent variable using the regression line. 


22.7 Statistical Inference and Hypothesis Testing 


Inferential statistics is concerned with statistical techniques to infer properties of a 
population from samples taken from the population. Often, it is infeasible or 
inconvenient to study all members of a population, and so the properties of a 
representative sample are studied and statistical techniques are used to generalize 
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Fig. 22.9 Regression line 
Table 22.7 Methods to obtain a regression line 
Methods Description 
Inspection This is the simplest method and involves plotting the data in a scatter graph 


and then drawing a line that best suits the data. (This is subjective and so it is 
best to draw the mean point and ensure the regression line passes through this 
point.) 

Semi-averages This involves splitting the data into two equal groups, then finding and 
drawing the mean point in each group and joining these points with a straight 
line (i.e. the regression line) 


Least squares The method of least squares is mathematical and involves obtaining the 
regression line where the sum of the squares of the vertical deviations of all 
the points from the line is minimal 


these properties to the population. A statistical experiment is carried out to gain 
information from the sample, and it may be repeated as many times as required to 
gain the desired information. Statistical experiments may be simple or complex. 

There are two main types of inferential statistics and these are estimating 
parameters and hypothesis testing. Estimating parameters is concerned with taking 
a Statistic from the sample (e.g. the sample mean or variance) and using it to make a 
statement about the population parameter (i.e. the population mean or variance). 
Hypothesis testing is concerned with using the sample data to answer research 
questions such as whether a new drug is effective in the treatment of a particular 
disease. A sample is not expected to perfectly represent the population, as sampling 
errors will naturally occur. 

A hypothesis is a statement about a particular population whose truth or falsity is 
unknown. Hypothesis testing is concerned with determining whether the values of 
the random sample from the population are consistent with the hypothesis. There 
are two mutually exclusive hypotheses: one of these is the null hypothesis Hp and 
the other is the alternate research hypothesis H,. The null hypothesis Ho is what the 
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researcher is hoping to reject, and the research hypothesis H, is what the researcher 
is hoping to accept. 

Statistical testing is employed to test the hypothesis, and the result of the test is 
that we either reject the null hypothesis (and therefore, accept the alternative 
hypothesis) or that we fail to reject it (i.e. we accept) the null hypothesis. The 
rejection of the null hypothesis means that the null hypothesis is highly unlikely to 
be true and that the research hypothesis should be accepted. 

Statistical testing is conducted at a certain level of significance, with the prob- 
ability of the null hypothesis Hp being rejected when it is true never greater than o. 
The value o is called the level of significance of the test, with « usually being 0.1, 
0.05 or 0.005. A significance level B may also be applied with respect to accepting 
the null hypothesis Hy when Hp is false. The objective of a statistical test is not to 
determine whether or not Ho is actually true, but rather to determine whether its 
validity is consistent with the observed data. That is, Hp should only be rejected if 
the resultant data is very unlikely if Hp is true. 

The errors that can occur with hypothesis testing include type 1 and type 2 
errors. Type | errors occur when we reject the null hypothesis when the null 
hypothesis is actually true. Type 2 errors occur when we accept the null hypothesis 
when the null hypothesis is false (Table 22.8). 

For example, an example of a false positive is where the results of a blood test 
come back positive to indicate that a person has a particular disease when in fact the 
person does not have the disease. Similarly, an example of a false negative is where 
a blood test is negative indicating that a person does not have a particular disease 
when in fact the person does. 

Both errors are potentially very serious, with a false positive generating major 
stress and distress to the recipient, until further tests are done that show that the 
person does not have the disease. A false negative is potentially even more serious, 
as early detection of a serious disease is essential to its treatment, and so a false 
negative means that valuable time is lost in its detection, which could be very 
serious. 

The terms o and f represent the level of significance that will be accepted, and « 
may or may not be equal to B. In other words, « is the probability that we will reject 
the null hypothesis when the null hypothesis is true, and B is the probability that we 
will accept the null hypothesis when the null hypothesis is false. 


Table 22.8 Hypothesis testing 


Action Hp true, H, false Hp false, H, true 

Reject H, Correct False Positive—Type 2 error 
P(Accept Ho|Ho false) = B 

Reject Ho False Negative—Type | error Correct 


P(Reject Ho|Ho true) = o 


380 22 Statistics 


Testing a hypothesis at the « = 0.05 level is equivalent to establishing a 95% 
confidence interval. For 99% confidence, « will be 0.01, and for 99.999% confi- 
dence, then « will be 0.00001. 

The hypothesis may be concerned with testing a specific statement about the 
value of an unknown parameter 9 of the population. This test is to be done at a 
certain level of significance, and the unknown parameter may, for example, be the 
mean or variance of the population. An estimator for the unknown parameter is 
determined, and the hypothesis that this is an accurate estimate is rejected if the 
random sample is not consistent with it. Otherwise, it is accepted. 

The steps involved in hypothesis testing include the following: 


1. Establish the null and alternative hypotheses. 

2. Establish error levels (significance). 

3. Compute the test statistics (often a f-test). 

4. Decide on whether to accept or reject the null hypothesis. 


The difference between the observed and expected test statistic and whether the 
difference could be accounted for by normal sampling fluctuations are the key to the 
acceptance or rejection of the null hypothesis. For more detailed information on 
Statistics, see [1, 2]. 


22.8 Review Questions 


. What is statistics? 

. Explain how statistics may be abused. 

What is a random sample? How may it be generated? 

. Describe the charts available for the presentation of statistical data. 
Explain how the average of a sample may be determined. 

. Explain sample variance and sample standard deviation. 

. Explain the difference between correlation and regression. 

. Explain the methods for obtaining the regression line from data. 

. What is hypothesis testing? 


CAOIDNRWNHE 


22.9 Summary 


Statistics is an empirical science that is concerned with the collection, organization, 
analysis, interpretation and presentation of data. Statistics are widely used for 
forecasting as well as for presenting trends. They allow the behaviour of a 
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population to be studied and inferences to be made about the population. These 
inferences may be tested to ensure their validity. 

It is often not possible to study the entire population, and instead, a represen- 
tative subset or sample of the population is chosen. This random sample is used to 
make inferences regarding the entire population, and it is essential that the sample 
chosen is indeed random and representative of the entire population. Otherwise, the 
inferences made regarding the entire population will be invalid due to the intro- 
duction of a selection bias. 

The data gathered from a statistical study is often raw, and the way the data is 
presented is important. It is useful to present the information in pictorial form, as 
this enables analysis to be done and conclusions to be drawn. Bar charts, his- 
tograms, pie charts, and trend graphs may be employed. 

Statistical measures are concerned with the basic analysis of the data to deter- 
mine the average of the data, as well as how spread out the data is. The term 
‘average’ generally refers to the arithmetic mean of a sample, but it may also refer 
to the statistical mode or median of the sample. 

An important characteristic of a sample is its distribution and the spread of each 
element from some measure of central tendency (e.g. the mean). The standard 
deviation is the most common way to measure dispersion, and it gives the average 
distance of each element in the sample from the arithmetic mean. 

Correlation and linear regression are techniques for exploring the relationship 
between two variables. Correlation is concerned with quantifying the strength of the 
relationship between two variables, whereas regression expresses the relationship 
between the variable in the form of an equation. 

Inferential statistics is concerned with statistical techniques to infer properties of 
a population from samples taken from the population. A hypothesis is a statement 
about a particular population whose truth or falsity is unknown. Hypothesis testing 
is concerned with determining whether the values of the random sample from the 
population are consistent with the hypothesis. 
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23.1 Introduction 


Probability is a branch of mathematics that is concerned with measuring uncertainty 
and random events, and it provides a precise way of expressing the likelihood of a 
particular event occurring. Probability is also used as part of everyday speech in 
expressions such as “It is likely to rain in the afternoon”, where the corresponding 
statement expressed mathematically might be “The probability that it will rain in the 
afternoon is 0.7”. 

The modern theory of probability theory has its origins in work done on the 
analysis of games of chance by Cardano in the sixteenth century, and it was 
developed further in the seventeenth century by Fermat and Pascal and refined in 
the eighteenth century by Laplace. It led to the classical definition of the probability 
of an event being 


P(Event) #4Favourable Outcomes 
vent) = 
#Possible Outcomes 


There are several definitions of probability such as the frequency interpretation 
and the subjective interpretation of probability. For example, if a geologist states 
that “there is a 70% chance of finding gas in a certain region”, then this statement is 
usually interpreted in two ways: 


— The geologist is of the view that over the long run, 70% of the regions whose 
environmental conditions are very similar to the region under consideration have 
gas. [Frequency Interpretation]. 

— The geologist is of the view that it is likely that the region contains gas and that 
0.7 is a measure of the geologist’s belief in this hypothesis. [Belief 
Interpretation]. 
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That is, according to the frequency interpretation, the probability of an event is 
equal to the long-term frequency of the event’s occurrence when the same process 
is repeated many times. 

According to the belief, interpretation probability measures the degree of belief 
about the occurrence of an event or in the truth of a proposition, with a probability 
of 1 representing the certain belief that something is true and a probability of 0 
representing the certain belief that something is false, with a value in between 
reflecting uncertainty about the belief. 

Probabilities may be updated by Bayes’ Theorem (see Sect. 23.2.2), where the 
initial belief is the prior probability for the event, and this may be updated to a 
posterior probability with the availability of new information (see Sect. 23.6 for a 
short account of Bayesian Statistics). 


23.2 Basic Probability Theory 


Probability theory provides a mathematical indication of the likelihood of an event 
occurring, and the probability lies between 0 and 1. A probability of 0 indicates that 
the event cannot occur whereas a probability of 1 indicates that the event is 
guaranteed to occur. If the probability of an event is greater than 0.5, then this 
indicates that the event is more likely to occur than not to occur. 

A statistical experiment is conducted to gain certain desired information, and the 
sample space is the set of all possible outcomes of an experiment. The outcomes are 
all equally likely if no one outcome is more likely to occur than another. An event E 
is a subset of the sample space, and the event is said to have occurred if the outcome 
of the experiment is in event E. 

For example, the sample space for the experiment of tossing a coin is the set of 
all possible outcomes of this experiment, i.e. head or tail. The event that the toss 
results in a tail is a subset of the sample space. 


S= {h,t} E= {t}. 


Similarly, the sample space for the gender of a newborn baby is the set of 
outcomes, i.e. the newborn baby is a boy or a girl. The event that the baby is a girl is 
a subset of the sample space. 


S={b,g} E= {g}. 


For any two events E and F of a sample space S, we can also consider the union 
and intersection of these events. That is, 


— EUFconsists of all outcomes that are in E or F or both. 

— En Ftusually written as EF) consists of all outcomes that are in both E and F. 

— E* denotes the complement of E with respect to S and represents the outcomes of 
S that are not in E. 
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If EF = 9), then there are no outcomes in both E and F, and so the two events E 
and F are mutually exclusive. Events that are mutually exclusive cannot occur at the 
same time (i.e. they cannot occur together). 

Two events are said to be independent if the occurrence (or not) of one of the 
events does not affect the occurrence (or not) of the other. Two mutually exclusive 
events cannot be independent, since the occurrence of one excludes the occurrence 
of the other. 

The union and intersection of two events can be extended to the union and 
intersection of a family of events E;, Eo, ... E, (ie. U7_,E; and M'_,E)). 


23.2.1 Laws of Probability 


The probability of an event E occurring is given by 


___ #Outcomes in Event E 
- # Total Outcomes (in S)° 


P(E) 


The laws of probability essentially state that the probability of an event is 
between 0 and 1 and that the probability of the union of a mutually disjoint set of 
events is the sum of their individual probabilities. The probability of an event E is 
zero if E is an impossible event, and the probability of an event E is one if it is a 
certain event (Table 23.1). 

The probability of the union of two events (not necessarily disjoint) is given by 


P(EUF) = P(E) + P(F) — P(EF). 


The complement of an event E is denoted by E* and denotes that event E does 
not occur. Clearly, S = EUE‘and E and E* are disjoint and so 


P(S) = P(EUE‘) = P(E) + P(E*) = 1 
= P(E°) = 1—P(E). 


Table 23.1 Axioms of probability 


Axiom _ Description 


1 PS) =1 

2) P(0) =0 

3 0< PE) <1 

4 For any sequence of mutually exclusive events E;, Ep, ... E,. (i.e. E;E; = @ where 


i # j), then the probability of the union of these events is the sum of their individual 
probabilities, i.e. 


P(UL Ei) = Dies P(E) 
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The probability of an event E occurring given that an event F has occurred is 
termed the conditional probability (denoted by P(E|F)) and is given by 


P(E|F) = = , where P(F) > 0. 


This formula allows us to deduce that 


P(EF) = P(E|F)P(F). 


Example (Conditional Probability) 
A family has two children. Find the probability that they are both girls given that 
they have at least one girl. 


Solution (Conditional Probability) 

The sample space for a family of two children is S = {(g,g), (g,b), bg), (b,b)}. The 
event E where is at least one girl in the family is given by E = :{(g,g), (g,b), b,g)}, and 
the event that G both are girls is G = {(g,g)}, so we will determine the conditional 
probability P(G|E) that both children are girls given that there is at least one girl in the 
family: 


P(EG) = P(G) = P(g, g) = 1/4. 


P(E) = 3/4. 
P(G|E) = a = f = If, 


Two events E, F are independent if the knowledge that F has occurred does not 
change the probability that E has occurred. That is, P(E|F) = P(E) and since P(E| 
F) = P(EF)/P(F), we have that two events E, F are independent if 


P(EF) = P(E)P(F). 


Two events E and F that are not independent are said to be dependent. 


23.2.2 Bayes’ Formula 


Bayes’ formula enables the probability of an event E to be determined by a 
weighted average of the conditional probability of E given that the event F occurred 
and the conditional probability of E given that F has not occurred: 


E=ENS =ENn(FUF) 
= EFUEF*. 
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P(E) = P(EF) + P(EF*) (since EF EF* = 9) 
(E|F)P(F) + P(E|F*)P(F*) 
= P(E|F)P(F) + P(E|F*)(1 — P(F)). 


We may also get another expression of Bayes’ formula from noting that 
P(F|E) = —— = ——. 


Therefore, P(EF) = P(F|E)P(E) = P(E|F)P(F), 


P(FIE)P(E) 
EE ay 

This version of Bayes’ formula allows the probability to be updated where the 
initial or preconceived belief (i.e. P(E)) is the prior probability for the event, and 
this may be updated to a posterior probability (i.e. P(E|F is the updated probability), 
with the new information or evidence (i.e. P(F)) and the likelihood that the new 
information leads to the event (i.e. P(FIE)). 


Example (Bayes’ Formula) 


A medical lab is 99% effective in detecting a certain disease when it is actually 
present, and it yields a false positive for 1% of healthy people tested. If 0.25% of 
the population actually have the disease, what is the probability that a person has 
the disease if the patient’s blood test is positive? 


Solution (Bayes’ Formula) 


Let T be the event that that the patient’s test result is positive, and D the event that the 
tested person has the disease. Then the desired probability is P(D|T) and is given by 


P(DT) P(T|D)P(D) 
P(DIT) = Scr) = BCRD)P(D) + PCTD PD) 
0.99 « 0.0025 


= = l . 
0.99 « 0.0025 + 0.01 * 0.9975 on 


The reason that only approximately 20% of the population whose test results are 
positive actually have the disease may seem surprising, but is explained by the low 
incidence of the disease (just one person out of every 400 tested will have the 
disease and the test will correctly confirm that 0.99 have the disease, but the test 
will also state that 399 * 0.01 = 3.99 have the disease and so the proportion of time 
that the test is correct is °°? /) 9943.99 = 0.1988). 
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23.3 Random Variables 


Often, some numerical quantity determined by the result of the experiment is of 
interest rather than the result of the experiment itself. These numerical quantities are 
termed random variables. A random variable is termed discrete if it can take on a 
finite or countable number of values, and otherwise, it is termed continuous. 

The distribution function (denoted by F(x)) of a random variable is the probability 
that the random variable X takes on a value less than or equal to x. It is given by 


F(x) = P{X <x}. 


All probability questions about X can be answered in terms of its distribution 
function F. For example, the computation of P {a < X < b} is given by 


P{a<X<b} = P{X <b} — P{X <a} 
= F(b) — F(a). 


The probability mass function for a discrete random variable X (denoted by p(a)) 
is the probability that the random variable is a certain value. It is given by 


p(a) = P{X = a}. 


Further, F(a) can also be expressed in terms of the probability mass function 


F(a) =P{X<a} = SO p(x). 


Vx<oa 


X is a continuous random variable if there exists a non-negative function f 
(x) (termed the probability density function) defined for all x € (— x, ox) such that 


P{X € B} = [ reas 


All probability statements about X can be answered in terms of its density 
function f(x). For example: 


b 
P{a<X<b}= [fea 


0.6) 


pte Cees aiS J fever 


—0o 
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The function f(x) is termed the probability density function, and the probability 
distribution function F(a) is defined by 


F(a) = P{X <a} = : f (x)dx; 


Further, the first derivative of the probability distribution function yields the 
probability density function. That is, 


“/aaF (4) = f(a). 


The expected value (i.e. the mean) of a discrete random variable X (denoted E 
[X]) is given by the weighted average of the possible values of X: 


>>, 4iP{X = x;} Discrete Random variable 


E[X] = J xf (x)ax Continuous Random variable 


—oo 


Further, the expected value of a function of a random variable is given by E[g 
(X)] and is defined for the discrete and continuous case, respectively. 


>>, g(m)P{X = x;} Discrete Random variable 
Elg(X)] = J g(x)f (x)dx Continuous Random variable ° 


The variance of a random variable is a measure of the spread of values from the 
mean and is defined by 


Var(X) = E[X?]—(E[X])’. 


The standard deviation o is given by the square root of the variance. That is, 


oO = »/Var(X). 


The covariance of two random variables is a measure of the relationship 
between two random variables X and Y and indicates the extent to which they both 
change (in either similar or opposite ways) together. It is defined by 


Cov(X, Y) = E[XY] — E[X] E[Y]. 


It follows that the covariance of two independent random variables is zero. 
Variance is a special case of covariance (when the two random variables are 
identical). This follows since Cov(X,X) = E[X.X|—(E[X])(E[X]) = E[X?]— 
(E[x])?= Var(X). 
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A positive covariance (Cov(X,Y) > 0) indicates that Y tends to increase as X 
does, whereas a negative covariance indicates that Y tends to decrease as X increases. 

The correlation of two random variables is an indication of the relationship 
between two variables X and Y (we discussed correlation and regression in 
Sect. 22.6). If the correlation is negative and close to —1, then Y tends to decrease 
as X increases, and if it is positive and close to 1, then Y tends to increase as X 
increases. A correlation close to zero indicates no relationship between the two 
variables; a correlation of r = —0.4 indicates a weak negative relationship; whereas 
a correlation of r = 0.8 indicates a strong positive relationship. The correlation 
coefficient is between +1 and is defined by 


Corr(X, Y) = __Cov(X% ¥) 
Var(X)Var(Y) | 


Once the correlation between two variables has been calculated, the probability 
that the observed correlation was due to chance can be computed. This is to ensure 
that the observed correlation is a real one and not due to a chance occurrence. 


23.4 Binomial and Poisson Distributions 


The binomial and Poisson distributions are two important distributions in statistics, 
and the Poisson distribution may be used as an approximation for the binomial. The 
binomial distribution was first used in games of chance, and it has the following 
characteristics: 


— The existence of a trial of an experiment, which is defined in terms of two states 
namely success or failure. 

— The identical trials may be repeated a number of times yielding several successes 
and failures. 

— The probability of success (or failure) is the same for each trial. 


A Bernouilli trial is where there are just two possible outcomes of an experi- 
ment, i.e. success or failure. The probability of success and failure is given by 


P{X = 1} =p; 
P{X =O} =1-p. 


The mean of the Bernouilli distribution is given by p_ (since 
E[X] = 1.p+0.(1—p) =p), and the variance is given by p(l-p) (since 
E[X?] — E[X]'= p — p® = p(1 — p)). 

The Binomial distribution involves n Bernouilli trials, where each trial is 
independent and results in either success (with probability p) or failure (with 
probability 1 - p). The binomial random variable X with parameters n and 
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p tepresents the number of successes in m independent trials, where X; is the result 
of i th trial and X is represented as 


X= yee 
i=1 


1 if the7thtrialisasuccess 
Xi; = : . 
0 otherwise 


The probability of i successes from n independent trials is then given by the 
binomial theorem: 


Pxed=(jpl=p 70a 


Clearly, E[X;] = p and Var(X;) = p(1 — p) (since X; is an independent Ber- 
nouilli random variable). The mean of the Binomial distribution E[X] is the sum of 
the mean of the E[X;,], i.e. LTE[X;] = np, and the variance Var(X) is the sum of the 
Var(X;) (since the X; are independent random variables) and _ so 
Var (X) = np(1 —p). The binomial distribution is symmetric when p = 0.5, and 
the distribution is skewed to the left or right when p 4 0.5 (Fig. 23.1). 


Example (Binomial Distribution) 

The probability that a printer will need correcting adjustments during a day is 
0.2. If there are five printers running on a particular day, determine the probability 
of the following: 


. No printers need correcting. 

. One printer needs correcting. 

. Two printers require correcting. 

. More than two printers require adjusting. 


BRwWND eRe 


Binomial Distribution (n=10, p=0.5) 


BB Series1 


0 1 2 3 4 5 6 7 8 8) 10 


X (Number of Successes) 


Fig. 23.1 Binomial distribution 
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Solution (Binomial Distribution) 
There are five trials (with n = 5, p = 0.2, and the success of a trial is a printer 
needing adjustments). And so, 


(1) This is given by P(X = 0) = (§) 0.2° « 0.8° = 0.3277. 

(2) This is given by P(X = 1) = (7) 0.2' « 0.84 = 0.4096. 

(3) This is given by P(X = 2) = (3) 0.2? « 0.8° = 0.205. 

(4) This is given by 1 — P(2 or fewer printers need correcting). 


= 1-[P(X = 0) + P(X = 1) + P(X =2) 
= 1 — [0.3277 + 0.4096 + 0.205] 

= 10.9423 

= 0.0577. 


The Poisson distribution may be used as an approximation to the Binomial 
Distribution when n is large (e.g. n > 30) and p is small (e.g. p < 0.1). The char- 
acteristics of the Poisson distribution are as follows: 


— The existence of events that occur at random and may be rare (e.g. road 


accidents). 
— An interval of time is defined in which events may occur. 


The probability of i successes (where i = 0, 1, 2, ...) is given by 


efi 


i! 


P(X =i) = 


The mean and variance of the Poisson distribution are given by i. 
Example (Poisson Distribution) 


Customers arrive randomly at a supermarket at an average rate of 2.5 customers per 
minute, where the customer arrivals form a Poisson distribution. Determine the 
probability that 


1. No customers arrive at any particular minute. 

2. Exactly one customer arrives at any particular minute. 
3. Two or more customers arrive at any particular minute. 
4. One or more customers atrive at any 30-s period. 


Solution (Poisson Distribution) 


The mean A is 2.5 per minute for parts 1-3 and A is 1.25 for part 4. 
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P(X = 0) =e? * 2.5°/0! = 0.0821. 

POH 1) e245 = 02052 

. P(2 or more) = 1—P(X = 0 or X = 1)|= 1—[P(K = 0) + P(X = 1)] = 0.7127. 

P(1 or more) = 1—P(X = 0) = 1 — e7!?5 « 1.25°/0! = 1—0.2865 = 0.7134. 


RYN 


23.5 The Normal Distribution 


The normal distribution is the most important distribution in statistics and it occurs 
frequently in practice. It is shaped like a bell and it is popularly known as the 
bell-shaped distribution, and the curve is symmetric about the mean of the distri- 
bution. The empirical frequencies of many natural populations exhibit a bell-shaped 
(normal) curve, such as the frequencies of the height and weight of people. The 
largest frequencies cluster around the mean and taper away symmetrically on either 
side of the mean. The German mathematician, Gauss (Fig. 23.2), originally studied 
the normal distribution, and it is also known as the Gaussian distribution. 

The normal distribution is a continuous distribution, and it has two parameters 
namely the mean p and the standard deviation o. It is a continuous distribution, and so 
it is not possible to find the probability of individual values, and thus, it is only 
possible to find the probabilities of ranges of values. The normal distribution has the 
important properties that 68.2% of the values lie within one standard deviation of the 
mean, with 95% of the values within two standard deviations and 99.7% of values are 
within 3 standard deviations of the mean. In other words, the value of the normal 
distribution is practically zero when the value of x is more than three standard 
deviations from the mean. The shaded area under the curve in Fig. 23.3 represents 
two standard deviations of the mean and comprises 95% of the population. 

The normal distribution N has mean y and standard deviation o. Its density 
function f(x) (where —oco <x<o00) is given by 


Fig. 23.2. Carl Friedrich 
Gauss 
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Fig. 23.3 Standard normal 
bell curve (Gaussian 
distribution) 


23.5.1 Unit Normal Distribution 


The unit (or standard) normal distribution Z(0,1) has a mean 0 and standard deviation 
of 1. Every normal distribution may be converted to the unit normal distribution by 
Z = (X — pn) /o, and every probability statement about X has an equivalent proba- 
bility statement about Z. The unit normal density function is given by 


1 1 
f(y) > Jor Ps 


The unit normal distribution is symmetric about 0 (as the mean is zero), and the 
process of converting a normal distribution with mean py and standard deviation ¢ is 
termed standardizing the x-value. There are tables of values that give the probability 
of a Z score between zero and the one specified. 


Example (Normal Distribution) 


Weights of bags of oranges are normally distributed with a mean of 3lbs and a 
standard deviation of 0.2 lb. The delivery to a supermarket is 350 bags at a time. 
Determine the following: 


1. Standardize to a unit normal distribution. 

2. What is the probability that a standard bag will weigh more than 3.5 lbs? 

3. How many bags from a single delivery would be expected to weigh more than 
3.5 Ibs. 


Solution (Normal Distribution) 
1. Z=X— p/o = X-3/ 0.2. 
2. Therefore, when X = 3.5, we have Z = 3.5 —-3 / 0.2 = 2.5. 


For Z = 2.5, we have from the unit normal tables that 
P(Z < 2.50) = 0.9938 = P(X <3.5). 
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Therefore, P(X > 3.5) = 1—P(X <3.5) = 1—0.9938 = 0.0062. 


3. The proportion of all bags that have a weight greater than 3.5 Ibs is 0.0062, and 
so it would be expected that there are 350 * 0.0062 = 2.17 bags with a 
weight >3.5, and so in practical terms, we would expect two bags to weigh more 
than 3.5 lbs. 


The normal distribution may be used as an approximation to the binomial when 
n is large (e.g. n > 30) and when p is not too small or large. This is discussed in the 
next section, where the mean of the normal distribution is np and the standard 


deviation is \/np(1 — p). 


23.5.2 Confidence Intervals and Tests of Significance 


The study of normal distributions helps in the process of estimating or specifying a 
range of values, where certain population parameters (such as the mean) lie from 
the results of small samples. Further, the estimate may be stated with a certain 
degree of confidence, such as there is 95 or 99% confidence that the mean value lies 
between 4.5 and 5.5. That is, confidence intervals (also known as confidence limits) 
specify a range of values within which some unknown population parameter lies 
with a stated degree of confidence, and it is based on the results of the sample. 

The confidence interval for an unknown population mean where the sample 
mean, sample variance, the sample size and desired confidence level are known is 
given by 


=! 
u 
TT 
NX 


In the formula, X is the sample mean, s is the sample standard deviation, n is the 
sample size and z is the confidence factor (for a 90% confidence interval, z = 1.64; 
for the more common 95% confidence interval, z = 1.96 and z = 2.58 for the 99% 
confidence interval). 


Example (Confidence Intervals) 


Suppose a new motor fuel has been tested on 30 similar cars, and the fuel con- 
sumption was 44.l1mpg with a standard deviation of 2.9 mpg. Calculate a 95% 
confidence interval for the fuel consumption of this model of car. 


Solution (Confidence Intervals) 


The sample mean is 44.1, the sample standard deviation is 2.9, the sample size is 30 
and the confidence factor is 1.96, so the 95% confidence interval is 


396 23 Probability Theory 


PR eaaiwen yy es Re vtec 
vn V30 
= 44.1 + 1.96 « 0.5295 
= 44.1 + 1.0378 


= (43.0622, 45.1378). 


That is, we can say with 95% confidence that the fuel consumption for this 
model of car is between 43.0 and 45.1 mpg. 

The confidence interval for an unknown population mean where the sample 
proportion and sample size are known is given by 


p(1—p) 


pxz 


In the formula, p is the sample proportion, n is the sample size and z is the 
confidence factor. 


Example (Confidence Intervals) 


Suppose 3 faulty components are identified in a random sample of 20 products 
taken from a production line. What statement can be made about the defect rate of 
all finished products? 


Solution (Confidence Intervals) 


The proportion of defective products in the sample is p = °/:) = 0.15, and the 
sample size is n = 20. Therefore, the 95% confidence interval for the population 
mean is given by 


pI —p) 


15(1—0.1 
ptz/———* = 0.15 + 1.96 ees 


20 


= 0.15 + 1.96 * 0.0798 
= 0.15 + 0.1565 
= (—0.0065, 0.3065). 


That is, we can say with 95% confidence that the defective rate of finished 
products lies between 0 and 0.3065. 


Tests of Significance for the Mean 


Tests of significance are related to confidence intervals and use the concepts from 
the normal distribution. To test whether a sample of size n, with sample mean X 

and sample standard deviation s, could be considered as having been drawn from 
a population with mean p, the test statistic must lie in the range —1.96 to 1.96. 
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That is, the test is looking for evidence of a significant difference between the 
sample mean X and the population mean y, and evidence is found if z lies outside of 
the stated limits, whereas if z lies within the limits, then there is no evidence that the 
sample mean is different from the population mean. 


Example (Tests of Significance) 

A new machine has been introduced and management is questioning whether it 
is more productive than the previous one. Management takes 15 samples of this 
week’s hourly output to test whether it is less productive, and the average pro- 
duction per hour is 1250 items with a standard deviation of 50. The output per hour 
of the previous machine was 1275 items per hour. Determine with a test of sig- 
nificance whether the new machine is less productive. 


Solution (Tests of Significance) 


The sample mean is 1250, the population mean is 1275, the sample standard 
deviation is 50 and the sample size is 15. 


—X—p 1250-1275 —25 | 
= ) ~ (3%) ~ 12.91 — 
vn v5 
This lies within the range —1.96 to 1.96 and so there is no evidence of any 


significant difference between the sample mean and the population mean, and so 
management is unable to make any statement on differences in productivity. 


0.1936. 


23.5.3. The Central Limit Theorem 


A fundamental result in probability theory is the Central Limit Theorem, which 
essentially states that the sum of a large number of independent and identically 
distributed random variables has a distribution that is approximately normal. That 
is, suppose X1,X5, ... X,, is a sequence of independent random variables each with 
mean j and variance o*. Then for large n, the distribution of 


XitXet-++X,— mp 
ay/n 
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is approximately that of a unit normal variable Z. One application of the central limit 
theorem is in relation to the binomial random variables, where a binomial random 
variable with parameters (n,p) represents the number of successes of n independent 
trials, where each trial has a probability of p of success. This may be expressed as 


X= Xi, +X2.+ ied + Xn, 


where X; = 1 if the ith trial is a success and is 0 otherwise. The mean of the 
Bernoulli trial E(X;) = p, and its variance is Var(X;) = p(1 — p). (The mean of the 
Binomial distribution with n Bernoulli trials is np and the variance is np(1 — p).) By 
applying the central limit theorem, it follows that for large n that 


X — np 
np(1 — p) 


will be approximately a unit normal variable (which becomes more normal as 
n becomes larger). 

The sum of independent normal random variables is normally distributed, and it 
can be shown that the sample average of X,,X, ... X,, is normal, with a mean equal 
to the population mean but with a variance reduced by a factor of 1/n. 


B(x) = 2) -y 


co 
a 


= 1 
Var(X) = oe Var(X;) = 
It follows from this that the following is a unit normal random variable. 


gta) 


o 

The term six-sigma (60) is a methodology concerned with continuous process 
improvement to improve business performance, and it aims to develop very high 
quality close to perfection. It was developed by Bill Smith at Motorola in the early 
1980s, and it was later used by leading companies such as General Electric. A 60 
process is one in which 99.9996% of the products are expected to be free from 
defects (3.4 defects per million) [1]. 

There are many other well-known distributions such as the hypergeometric 
distribution that describes the probability of i successes in n draws from a finite 
population without replacement, the uniform distribution, the exponential distri- 
bution and the gamma distribution. The mean and variance of these distributions are 
summarized in Table 23.2 below. 
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Table 23.2 Probability distributions 


Distribution Density function Mean/variance 

name 

Hypergeometric P{X =i} = (N;)(M,_,)/(N, *M) nN/N+M,np(1 — p)[1-(n— 1)/N+M - ]] 
Uniform f(x) = 1/(B—a)a<x<B, 0 (a+ B)/2, (B—a)?/12 

Exponential feo) Shes 1/d,1 [i 

eae F(x) = eax)" 1/T (a) at/A, 0/2? 


23.6 Bayesianism 


Bayesian thinking is named after Thomas Bayes who was an eighteenth-century 
English theologian and statistician, and it differs from the frequency interpretation 
of probability in that it considers the probability of an event to be a measure of 
one’s personal belief in the event. According to the frequentist approach, only 
repeatable events such as the result from flipping a coin have probabilities, where 
the probability of an event is the long-term frequency of occurrence of the particular 
event. Bayesians view probability in a more general way and probabilities may be 
used to represent the uncertainty of an event or hypothesis. It is perfectly acceptable 
in the Bayesian view of the world to assign probabilities to non-repeatable events, 
whereas a strict frequentist would claim that such probabilities do not make sense, 
as they are not repeatable. 

Bayesianism provides a way of dealing rationally with randomness and risk in 
daily life, and it is very useful when the more common frequency interpretation is 
unavailable or has limited information. It interprets probability as a measure of 
one’s personal belief in a proposition or outcome, and it is essential to first use all 
your available prior knowledge to form an initial estimate of the probability of the 
event or hypothesis. Further, when reliable frequency data becomes available, the 
measure of personal belief would be updated accordingly to equal the probability 
calculated by the frequency calculation. Further, the probabilities must be updated 
in the light of new information that becomes available, as probabilities may change 
significantly from new information and knowledge. Finally, no matter how much 
the odds move in your favour, there is eventually one final outcome (which may or 
may not be the desired event). 

Often, in an unreliable and uncertain world, we base our decision-making on a 
mixture of reflection and our gut instinct (which can be wrong). Often we encounter 
several constantly changing random events and so it is natural to wonder about the 
extent to which rational methods may be applied to risk assessment and 
decision-making in an uncertain world. 

An initial estimate is made of the belief in the proposition, and if you always rely 
on the most reliable and objective probability estimates while keeping track of 
possible uncertainties and updating probabilities in line with new data, then the final 
probability number computed will be the best possible. 
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We illustrate the idea of probabilities being updated with an adapted excerpt 
from a children’s story called “Fortunately”, which was written by Remy Charlip in 
the 1960s [2]: 


— A lady went on a hot air balloon trip. 

nfortunately, she fell out. 

ortunately, she had a parachute on. 

nfortunately, the parachute did not open. 

ortunately, there was a haystack directly below. 

nfortunately, there was a pitchfork sticking out at the top of the haystack. 
ortunately, she missed the pitchforks. 

nfortunately, she missed the haystack. 


| 
qCmamcanad 


The story illustrates how probabilities can change dramatically based on new 
information, and despite all the changes to the probabilities during the fall, the final 
outcome is a single result (i.e. either life or death). Let p be the probability of 
survival, then the value of p changes as she falls through the sky based on new 
information at each step. Table 23.3 illustrates an estimate of what the probabilities 
might be. 

However, even if probability calculations become irrelevant after the event, they 
still give the best chances over the long term. Over our lives, we make many 
thousands of decisions about where and how to travel, what diet we should have 
and so on, and though the impact of each of these decisions on our life expectancy 
is very small, their combined effects are potentially significant. Clearly, careful 
analysis is needed for major decisions rather than just making a decision based on 
gut instinct. 

For the example above, we could estimate probabilities for the various steps 
based on the expectation of probability of survival on falling without a parachute 
and the expectation of probability of survival on falling onto a haystack without a 
parachute, and we would see wildly changing probabilities from the changing 
circumstances. 


Table 23.3 Probability of survival 


Step Prob. survival 
A lady went on a hot air balloon trip p = 0.999998 
Unfortunately, she fell out p = 0.000001 
Fortunately, she had a parachute on p = 0.999999 
Unfortunately, the parachute did not open p = 0.000001 
Fortunately, there was a haystack directly below p= 0.5 
Unfortunately, there was a pitchfork sticking out at the top of the haystack =p = 0. 000,001 
Fortunately, she missed the pitchforks p= 0.5 
Unfortunately, she missed the haystack p = 0.000001 
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We discussed Bayes’ formula in Sect. 23.2.2, which allows the probability to be 
updated where the initial or preconceived belief (i.e. P(E) is the prior probability for 
the event), and this may be updated to a posterior probability (i.e. P(E|F is the 
updated probability), with the new information or evidence (i.e. P(F)) and the 
likelihood that the new information leads to the event (i.e. P(F|E)). 


23.7 Queueing Theory 


The term ‘queue’ refers to waiting in line for a service, such as waiting in line at a 
bakery or a bank, and queueing theory is the mathematical study of waiting lines or 
queues. The origins of queueing theory are in work done by Erlang at the 
Copenhagen Telephone Exchange in the early twentieth century where he modelled 
the number of telephone calls arriving as a Poisson process. 

Queueing theory has been applied to many fields including telecommunications 
and traffic management. This section aims to give a flavour and a very short 
introduction to queueing theory, and it has been adapted from [3]. The interested 
reader may consult the many other texts available for more detailed information 
(e.g. [4)). 

A supermarket may be used to illustrate the ideas of queueing theory, as it has a 
large population of customers some of whom may enter the supermarket and 
queueing system (the checkout queues). Customers will generally wait for a period 
of time in a queue before receiving service at the checkout, and they wait for a 
further period of time for the actual service to be carried out. Each service facility 
(the checkouts) contains identical servers, and each server is capable of providing 
the desired service to the customer (Fig. 23.4). 


Population 
of customers 


RRA 


Queue of 
customers 


Fig. 23.4 Basic queueing system 
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Clearly, if there are no waiting lines, then immediate service is obtained. 
However, in general, there are significant costs associated with the provision of an 
immediate service, and so there is a need to balance costs with a certain amount of 
waiting. 

Some queues are bounded (i.e. they can hold only a fixed number of customers), 
whereas others are unbounded and can grow as large as is required to hold all 
waiting customers. The customer source may be finite or infinite, and where the 
customer source is finite but very large, it is often considered to be infinite. 

Random variables (described by probability distribution functions) arise in 
queueing problems, and these include the random variable g, which represents the 
time that a customer spends in the queue waiting for service; the random variable s, 
which represents the amount of time that a customer spends in service and the 
random variable w, which represents the total time that a customer spends in the 
queueing system (Fig. 23.5). Clearly, 


w=qts. 


It is assumed that the customers arrive at a queueing system one at a time at 
random times (fo < ft; < ... <¢,) with the random variable t% = t — t— 4 repre- 
senting the interarrival times (i.e. it measures the times between successive arri- 
vals). It is assumed that these random variables are independent and identically 
distributed, and it is usually assumed that arrival form a Poisson arrival process. 

A Poisson arrival process is characterized by the fact that the interarrival times 
are distributed exponentially. That is, 


P(t<t) =1-e™. 


Fig. 23.5 Sample random 
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Further, the probability that exactly n customers will arrive in any time interval 
of length ¢ is given by 


egy 


(where n = 0,1,2,....) 
n! 


? 


where 2 is a constant average arrival rate of customers per unit time, and the 
number of arrivals per unit time is Poisson distributed with mean i. 

Similarly, it is usual to assume in queueing theory that the service times are 
random with p denoting the average service rate, and we let s, denote the service 
time that the kth customer requires from the system. The distribution of service 
times is given by 


W,(t) = P(s<t) =1-e™. 


The capacity of the queues may be infinite (where every arriving customer is 
allowed to enter the queueing system no matter how many waiting customers are 
present) or finite (where arriving customers may wait only if there is still room in 
the queue). 

Queueing systems may be single server (one server serving one customer at a 
time) systems or multiple servers (several identical servers that can service c cus- 
tomers at a time). The method by which the next customer is chosen from the queue 
to be serviced is termed the queue discipline, and the most common method is first 
come, first served (FCFS). Other methods include the last in, first out (LIFO), the 
shortest job first or the highest priority job next. 

Customers may exhibit various behaviours in a queueing system such as 
deciding not to join a queue if it is too long, switching between queues to try to 
obtain faster service or leaving the queueing system if they have waited too long. 
There are many texts on queueing theory, and for a more detailed account of 
queueing theory, see [4]. 

The reader is referred to [5, 6] for a more detailed account of probability and 
Statistics. 


23.8 Review Questions 


What is probability? 

Explain the laws of probability. 

What is a sample space? What is an event? 

Prove Boole’s inequality P( U Ey) < }*7_, P(E;) where the E; are not 
necessarily disjoint. 

5. A couple has two children. What is the probability that both are girls if 
the eldest is a girl? 


get a 
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6. What is a random variable? 

7. Explain the difference between the probability mass function and the 
probability density function (for both discrete and continuous random 
variables). 

8. Explain variance, covariance and correlation. 

9. What is the binomial distribution and what is its mean and variance? 

10. What is the Poisson distribution and what is its mean and variance? 

11. What is the normal distribution and what is its mean and variance? 

12. What is the unit normal distribution and what is its mean and variance? 
13. Explain the significance of the central limit theorem. 

14. What is Bayes’ theorem? Explain the importance of Bayesian thinking. 


23.9 Summary 


Probability is a branch of mathematics that is concerned with measuring uncertainty 
and random events, and it provides a precise way of expressing the likelihood of a 
particular event occurring, and the probability is a numerical value between 0 and 1. 
A probability of 0 indicates that the event cannot occur, whereas a probability of 1 
indicates that the event is guaranteed to occur. If the probability of an event is 
greater than 0.5, then this indicates that the event is more likely to occur than not to 
occur. 

A sample space is the set of all possible outcomes of an experiment, and an event 
E is a subset of the sample space, and the event is said to have occurred if the 
outcome of the experiment is in the event E. Bayes’ formula enables the probability 
of an event E to be determined by a weighted average of the conditional probability 
of E given that the event F occurred and the conditional probability of E given that 
F has not occurred. 

Often, some numerical quantity determined by the result of the experiment is of 
interest rather than the result of the experiment itself. These numerical quantities are 
termed random variables. The distribution function of a random variable is the 
probability that the random variable X takes on a value less than or equal to x. 

The binomial and Poisson distributions are important distributions in statistics, 
and the Poisson distribution may be used as an approximation for the binomial. The 
Binomial distribution involves n Bernouilli trials, where each trial is independent 
and results in either success or failure. The mean of the Bernouilli distribution is 
given by p and the variance by p(1 — p). 

The normal distribution is a continuous distribution, and the curve is symmetric 
about the mean of the distribution. It has two parameters namely the mean py and the 
standard deviation o. Every normal distribution may be converted to the unit 
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normal distribution by Z = (X — 1) /o, and every probability statement about X 
has an equivalent probability statement about the unit distribution Z. 

The Central Limit Theorem essentially states that the sum of a large number of 
independent and identically distributed random variables has a distribution that is 
approximately normal. Bayesianism provides a way of dealing rationally with 
randomness and risk in daily life, and it interprets probability as a measure of one’s 
personal belief in a proposition or outcome. 

Queueing theory is the mathematical study of waiting lines or queues, and its 
origins are in work done by Erlang in the early twentieth century. Customers will 
generally wait for a period of time in a queue before receiving service, and they wait 
for a further period of time for the actual service to be carried out. Each service 
facility (the checkouts) contains identical servers, and each server is capable of 
providing the desired service to the customer. Queueing theory has been applied to 
many fields including telecommunications and traffic management. 
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24.1 Introduction 


Operations research is a multi-disciplinary field that is concerned with the appli- 
cation of mathematical and analytic techniques to assist in decision-making. It 
employs techniques such as mathematical modelling, statistical analysis and 
mathematical optimization as part of its goal to achieve optimal (or near-optimal) 
solutions to complex decision-making problems. The modern field of operations 
research includes other disciplines such as computer science, industrial engineering, 
business practices in manufacturing and service companies, supply chain man- 
agement and operations management. 

Pascal did early work on operations research in the seventeenth century. He 
attempted to apply early work on probability theory to solve complex 
decision-making problems. Babbage’s work on the transportation and sorting of 
mail contributed to the introduction of the uniform “Penny Pos?’ in England in the 
nineteenth century. The origins of the operations research field are from the work of 
military planners during the First World War, and the field took off during the 
Second World War as it was seen as a scientific approach to decision-making using 
quantitative techniques. It was applied to strategic and tactical problems in military 
operations, where the goal was to find the most effective utilization of limited 
military resources through the use of quantitative techniques. It played an important 
role in solving practical military problems such as determining the appropriate 
convoy size in the submarine war in the Atlantic. 

Numerous peacetime applications of the field of operations research emerged 
after the Second World War, where operations research and management science 
were applied to many industries and occupations. It was applied to procurement, 
training, logistics and infrastructure in addition to its use in operations. The pro- 
gress that has been made in the computing field means that operations research can 
now solve problems with thousands of variables and constraints. 
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Operations research (OR) is the study of mathematical models for complex 
organizational systems, where a model is a mathematical description of a system 
that accounts for its known and inferred properties, and it may be used for the 
further study of its properties. A system is a functionally related collection of 
elements such as a network of computer hardware and software. Optimization is a 
branch of operations research that uses mathematical techniques to derive values 
from system variables that will optimize system performance. 

Operations research has been applied to a wide variety of problems including 
network optimization problems, designing the layouts of the components on a 
computer chip, supply chain management, critical path analysis during project 
planning to identify key project activities that affect the project timeline, scheduling 
project tasks and personnel and so on. Several of the models used in operations 
research are described in Table 24.1 below. 


Table 24.1 Models used in operations research 


Model Description 
Linear These problems aim to find the best possible outcome (where this is 
programming expressed as a linear function) such as to maximize a profit or 


minimize a cost subject to various linear constraints. The function and 
constraints are linear functions of the decision variables, and modern 
software can solve problems containing millions of variables and 
thousands of constraints 


Network flow This is a special case of the general linear programming problem and 

programming includes problems such as the transportation problem, the shortest path 
problem, the maximum flow problem and the minimum cost problem. 
There are very efficient algorithms available for these 


Integer This is a special case of the general linear programming problem, 
Programming where the variables are required to take on discrete values 
Non-linear The function and constraints are non-linear, and these are much more 
programming difficult to solve than linear programming. Many real-world 


applications require a non-linear model, and often solution is 
approximated with a linear model 


Dynamic A dynamic programming (DP) model describes a process in terms of 

programming states, decisions, transitions and a return. The process begins in some 
initial state, a decision is made leading to a transition to a new state, 
and the process continues through a sequence of states until the final 
state is reached. The problem is to find a sequence that maximizes the 
total return 


Stochastic processes A stochastic process models practical situations where the attributes of 
a system randomly change over time (e.g. number of customers at an 
ATM machine, the share price), and the state is a snapshot of the 
system at a point in time that describes its attributes. Events occur that 
change the state of the system 


(continued) 
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Table 24.1 (continued) 


Model 


Markov chains 


Markov processes 


Game theory 


Simulation 


Time series and 
forecasting 


Inventory theory 


Reliability theory 
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Description 


A stochastic process that can be observed at regular intervals (such as 
every day or every week) can be described by a matrix, which gives the 
probabilities of moving to each state from every other state in one time 
interval. The process is called a Markov Chain when this matrix is 
unchanging over time 

A Markov process is a continuous-time stochastic process in which the 
duration of all state-changing activities are exponentially distributed 
Game theory is the study of mathematical models of strategic 
interaction among rational decision makers. It is concerned with the 
logical decision-making by humans, animals and computers 
Simulation is a general technique for estimating statistical measures of 
complex systems 


A time series is a sequence of observations of a periodic random 
variable and is generally used as input to an operation research decision 
model 


Aims to optimize inventory management, e.g. determining when and 
how much inventory should be ordered 


Aims to model the reliability of a system from probability theory 


Mathematical programming involves defining a mathematical model for the 
problem and using the model to find the optimal solution. A mathematical model 
consists of variables, constraints, the objective function to be maximized or mini- 
mized and the relevant parameters and data. The general form is 


Min or Max f(x), x2, .. Xn) (Objective function) 


g(X1,%2,-- 


xEx 


Xn) <(or >, >,= <)b; (Constraints) 


f,g are linear and X is continuous(for linear programming LP). 


A feasible solution is an assignment of values to the variables such that the 
constraints are satisfied. An optimal solution is one whose objective function 
exceeds all other feasible solutions (for maximization optimization). We now dis- 
cuss linear programming in more detail. 


24.2 Linear Programming 


Linear programming (LP) is a mathematical model for determining the best possible 
outcome (e.g. maximizing profit or minimizing cost) of a particular problem. The 
problem is subject to various constraints such as resources or costs, and the con- 
straints are expressed as a set of linear equations and linear inequalities. The best 
possible outcome is expressed as a linear equation. For example, the goal may be to 
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determine the number of products that should be made to maximize profit subject to 
the constraint of limited available resources. 

The constraints for the problem are linear, and they specify regions that are 
bounded by straight lines. The solution will lie somewhere within the regions 
specified, and a feasible region is a region where all of the linear inequalities are 
satisfied. Once the feasible region is found, the challenge is then to find where the 
best possible outcome may be maximized in the feasible region, and this will 
generally be in a corner of the region. The steps involved in developing a linear 
programming model include the following: 


— Formulation of the problem. 
— Solution of the problem. 
— Interpretation of the solution. 


Linear programming models seek to select the most appropriate solution from 
the alternatives that are available subject to the specified constraints. Often, 
graphical techniques are employed to sketch the problem and the regions corre- 
sponding to the constraints. 

The graphical techniques identify the feasible region where the solution lies, and 
then the maximization or minimization function is employed within the region to 
search for the optimal value. The optimal solution will lie at one or more of the 
comer points of the feasible region. 


24.2.1 Linear Programming Example 


We consider an example in an industrial setting where a company is trying to 
decide how many of each product it should make to maximize profits subject to the 
constraint of limited resources. 

Square Deal Furniture produces two products namely chairs and tables, and it 
needs to decide on how many of each to make each month in order to maximize 
profits. The amount of time to make tables and chairs and the maximum hours 
available to make each product, as well as the profit contribution of each product, 
are summarized in Table 24.2 below. There are additional constraints that need to 
be specified as follows: 


— At least 100 tables must be made. 
— The maximum number of chairs to be made is 450. 


Table 24.2 Square Deal Table Chair Hours available 
Furniture 

Carpentry 3h 4h 2400 

Painting 2h lh 1000 


Profit contribution 7 Euros 5 Euros 
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We use variables to represent tables and chairs and formulate an objective 
function to maximize profits subject to the constraints. 


T = Number of Tables to make. 
C = Number of Chairs to make. 

The objective function (to maximize profits) is then specified as 
Maximize the value of 5C+7T. 


The constraints on the hours available for carpentry and painting may be 
specified as follows: 


3T+4C<2400 (carpentry time available). 
2T+C< 1000 (painting time available). 


The constraints that at least 100 tables must be made and the maximum number 
of chairs to be made is 450 may be specified as follows: 


T>100 (number of tables). 
C<450 (number of chairs). 


Finally, it is not possible to produce a negative number of chairs or tables and 
this is specified as follows: 


T>0 (non-negative). 
C>0 (non-negative). 
The model is summarized as follows: 
Max 5C+7T (Maximation problem). 


3T+4C < 2400 (carpentry time available) 
2T+C< 1000 (painting time available) 


T> 100 (number of tables) 
C<450 (number of chairs) 
T>0 (non-negative) 
C>0 (non-negative) 


We graph the LP model and then use the graph to find a feasible region for 
where the solution lies, and we then identify the optimal solution. The feasible 
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region is an area where all of the constraints for the problem are satisfied, and the 
optimal solution lies at one or more of the corner points of the feasible region. 

First, for the constraints on the hours available for painting and carpentry 
3T+4C < 2400 and 2T+C< 1000, respectively, we draw the two _ lines 
3 T + 4C = 2400 and 2 T + C = 1000. We choose two points on each line and 
then join both points to form the line, and we choose the intercepts of both lines as 
the points. 

For the line 3T + 4C = 2400 when T is 0 C is 600 and when C is OT is 800. 
Therefore, the points (0, 600) and (800, 0) are on the line 3T + 4C = 2400. For the 
line 2T + C = 1000 when T = 0 then C = 1000 and when C = 0 then T = 500. 
Therefore, the points (0, 1000) and (500, 0) are on the line 2T + C = 1000. 

Figure 24.1 is the first step in developing a graphical solution and we note that 
for the first two constraints 3T + 4C < 2400 and 2T + C< 1000 the solution lies 
somewhere in the area bounded by the lines 3T + 4C = 2400, 2T + C = 1000, the 
T-axis and the C-axis. 

Next, we add the remaining constraints (T > 100,C < 450,T > 0,C > 0) 
to the graph, and this has the effect of reducing the size of the feasible region in 
Fig. 24.1 (which placed no restrictions on T and C). The feasible region can be 
clearly seen in Fig. 24.2, and the final step is to find the optimal solution in the 
feasible region that maximizes the profit function 5C + 7T. 

Figure 24.3 shows how we find the optimal solution by drawing the line 
7T + 5C =k within the feasible region, and this forms a family of parallel lines 
where the slope of the line is —5/, and k represents the profit. Each point in the 
feasible region is on one of the lines in the family, and to determine the equation of 
that line, we just input the point into the equation 7T + 5C = k. For example, the 
point (200, 0) is in the feasible region and it satisfies the equation 
TT + 5C = 7*200 + 5*0 = 1400. 


Fig. 24.1 Linear 
programming—developing a 
graphical solution 1000 


3T + 4C= 2400 


2T + C= 1000 
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Fig. 24.2 Feasible region of C 
solution 


T= 100 


2T + C= 1000 


3T + 4C= 2400 


Feasible 
Region 


Fig. 24.3 Optimal solution Cc 
T= 100 


\ + C= 1000 


TT+5C = 2800S 
TT+5C = 1400)\ 
3T + 4C= 2400 


Feasible 
Region 


We seek to maximize k and it is clear that the value of k that is maximal is at one 
of the corner points of the feasible region. This is the point of intersection of the 
lines 2T + C = 1000 and 3T + 4C = 2400, and we solve for T and C to get 
T = 320 and C = 360. This means that the equation of the line containing the 
optimal point is 7T + 5C = 2240 + 1800 = 4040. That is, its equation is 
TT + 5C = 4040 and so the maximum profit is €4040. 
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24.2.2 General Formulation of LP Problem 


The more general formulation of the linear programming problem can be stated as 
follows. Find variables x,, x2, ..., x, to optimize (i.e. maximize or minimize) the 
linear function: 


Z = ix, + 2X2 + +++ CnXny 


where the problem is subject to the linear constraints 


411%, +a2X12 + +++, ayjxjt +++ +Ainma(< = > )by 
An) X1 + GynXq + +++, GajxXj + +++ + Arnin(< = 2) dg 


Am1X1 + Am2X12 + +++ > AnjXj app + AmnXn( <=2 Dn 


and to non-negative constraints on the variables such as 


X1,X2,---5X, > 0, 


where aj, b; and c; are constants and x; are variables. 

The variables x, x2, ..., X, whose values are to be determined are called decision 
variables. 

The coefficients c;, C2, ...., C, are called cost (profit) coefficients. 

The constraints b,, bo, ...., Db, are called the requirements. 

A set of real values (x1, x2, ...., X,) which satisfies the constraints (including the 
non-negative constraints) is said to be a feasible solution. 

A set of real values (x, x2, ...., X,) which satisfies the constraints (including the 
non-negative constraints) and optimizes the objective function is said to be an 
optimal solution. 

There may be no solution, a unique solution or multiple solutions. 

The constraints may also be formulated in terms of matrices as follows: 


x] by 

411 412 43° «++ Gn b 

a a a ay “ ‘ 

= oe ii tee i n Pa by 

31 32 433 3n mies) 

Ami Gm2  Gm3 +++ Amn - i 
This may also be written as AX (< = >} )B and the optimization function 
may be written as Z = CX where C = (cy, Co, ..., Cy) and X = (xy, Xo, ..., X,)) and 


xX > 0. 
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Linear programming problems may be solved by graphical techniques (when 
there are a small number of variables) or analytic techniques using matrices. There 
are techniques that may be employed to find the solution of the LP problem that are 
similar to finding the solution to a set of simultaneous equations using Gaussian 
elimination. Matrices were discussed in Chap. 29. 


24.3 Cost-Volume-Profit Analysis 


A key concern in business is profitability, and management needs to decide on the 
volume of products to produce, including the costs and total revenue. Cost—vol- 
ume-profit analysis (CVPA) is a useful tool in the analysis of the relationship 
between the costs, volume, revenue and profitability of the products produced. The 
relationship between revenue and costs at different levels of output can be displayed 
graphically, with revenue behaviour and cost behaviour shown graphically. 

The breakeven point (BP) is where the total revenue is equal to the total costs, 
and breakeven analysis is concerned with identifying the volume of products that 
need to be produced to break even. 


Example (CVPA) 


Pilar is planning to set up a business that makes pottery cups, and she has been 
offered a workshop to rent for €800 per month. She estimates that she needs to 
spend €10 on the materials to make each pottery cup and that she can sell each cup 
for €25. She estimates that if she is very productive that she can make 500 pottery 
cups in a month. 

Prepare a table that shows the profit or loss that Pilar makes based on the sales of 
0, 100, 200, 300, 400 and 500 pottery cups. 


Solution (CVPA) 


Each entry in the table consists of the revenue for the volume sold, the material 
costs per volume of the pottery cups, the fixed cost of renting the workshop per 
month, the total cost per month and the net income per month (Table 24.3). 

The total sales (revenue) is determined from the volume of sales multiplied by 
the unit sales price of a pottery cup (€25). There are two types of costs that may be 
incurred namely fixed costs and variable costs. 


Table 24.3 Projected profit or loss per volume 


#Cups 0 100 200 300 400 500 
Revenue (sales) 0 2500 5000 7500 10,000 12,500 
Materials (var cost) 0 1000 2000 3000 4000 5000 
Workshop (fix cost) 800 800. 800 800 800. 800 
Total cost 800 1800 2800 3800 4800 5800 


Net income —€800 €700 €2200 €3700 €5200 €6700 
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Table 24.4 Revenue and 
costs 


Ttem 

Total revenue (TR) 

Total variable cost (TVC) 
Fixed cost (FC) 

Total cost (TC) 


Net income (profit) 
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Amount 
SIP XC 
VC *X 
FC 
FC+ TVC 
= FC+(VC « X) 
TR-TC 
(SP * X)—FC—(VC x X) 


Fixed costs are incurred irrespective of the volume of items produced, and so the 
cost of renting the workshop is a fixed cost. Variable costs are constant per unit of 
output and include the direct material and labour costs, and so the total variable cost 
increases as the volume increases. That is, the total variable cost is directly related 
to the volume of items produced, and Table 24.4 summarizes the revenue and costs. 

We may represent the relationships between volume, cost and revenue graphi- 
cally and use it to see the relationship between revenue and costs at various levels of 
output (Fig. 24.4). We may then use the graph to determine the breakeven point, for 


when total revenue is equal to the total cost. 


We may also determine the breakeven point algebraically by letting X represent 
the volume of cups produced for breakeven. Then breakeven is when the total 


revenue is equal to the total cost. That is, 


Fig. 24.4 Breakeven point 


100 200 300 400 500 


loss 
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SP * X = FC+ VC x X 
=> 25X = 800+ 10X 
=> 15X = 800 > X 


_ 800); = 53.3 units. 


The breakeven amount in revenue is 25 * 53.3 = €1333.32. 

Next, we present an alternate way of calculating the breakeven point in terms of 
contribution margin and sales. Contribution margin is the monetary value that each 
extra unit of sales makes towards profitability, and it is given by the selling price 
per unit minus the variable cost per unit. 

Each additional pottery cup sold increases the revenue by €25 whereas the 
increase in costs is just €10 (the materials required). That is, the contribution margin 
per unit is the selling price minus the variable cost per unit (i.e. SP - VC = €15), 
and so the total contribution margin is the total volume of units sold multiplied by 
the contribution margin per unit (i.e. X « (SP — VC) = 15X). 

The breakeven volume is reached when the total contribution margin covers the 
fixed cost (i.e. the cost of renting the workshop which is €800 is covered by the 
total contribution). That is, the breakeven volume is reached when 


FC 
X«(SP— VC) =FC-X= X= OM asin gee 
Example 


Suppose that the rent of the workshop is increased to €1400 per month and that it 
also costs €13 (more than expected) to make each cup and that she can sell each cup 
for just €20. What is her new breakeven volume and revenue? 


Solution 

The breakeven volume is reached when 
X « (SP — VC) = FC 
X « (20-13) = 1400 
7X = 1400 
X = 200. 


Further, the breakeven revenue is 


X *SP 
= 200 * 20 


= €4000. 
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24.4 Game Theory 


Game theory is the study of mathematical models of strategic interaction among 
rational decision makers, and it was originally applied to zero sum games where the 
gains or losses of each participant are exactly balanced by those of the other 
participants. 

The modern game theory emerged as a field following John von Neumann’s 
1928 paper on the theory of games of strategy [1]. The Rand corporation investi- 
gated possible applications of game theory to global nuclear strategy in the 1950s. 
Game theory has been applied to many areas including economics, biology and the 
social sciences. It is an important tool in situations where a participant’s best 
outcome depends on what other participants do, and their best outcome depends on 
what he/she does. We illustrate the idea of game theory through the following 
example. 


Example (Game Theory) 


We consider an example of two television networks that are competing for an 
audience of 100 million on the 8 to 9 p.m. time nighttime television slot. The 
networks announce their schedule ahead of time, but do not know the other net- 
work’s decision until the program begins. A certain number of viewers will watch 
Network 1, with the remainder watching Network 2. Market research has been 
carried out to show the expected number of viewers for each network based on what 
will be shown by the networks (Table 24.5). 


Problem to Solve (Viewing Figures) 

Determine the best strategy that both networks should employ to maximize their 
viewing figures. 

Table 24.5 shows the number of viewers of Network 1 for each type of film that 
also depends on the type of film that is being shown by Network 2. For example, if 
Network | is showing a western while Network 2 is showing a comedy, then 
Network 1 will have 60 million viewers, and Network 2 will have 100 — 60 = 40 
million viewers. However, if Network 2 was showing a soap opera, then the 
viewing figures for Network | are 15 million, and 100 — 15 = 85 million will be 
tuned into Network 2. 


Solution (Game Theory) 


Network 1 is a row player whereas Network 2 is a column player, and the table 
above is termed a payoff matrix. This is a constant-sum game (as the outcome for 
both players always adds up to a constant 100 million). 


Table 24.5 Network Nei D 

viewing figures 
Network | Western Soap Opera Comedy 
Western 35 15 60 
Soap opera 45 58 50 


Comedy 38 14 70 
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The approach to finding the appropriate strategy for Network 1 is to examine 
each option. If Network 1 decides to show a western, then it can get as many as 60 
million viewers if Network 2 shows a comedy or as few as 15 million viewers if 
Network 2 shows a soap opera. That is, this choice cannot guarantee more than 15 
million viewers. Similarly, if Network 1 shows a soap opera, it may get as many as 
58 million viewers if Network 2 shows a soap opera as well or as few as 45 million 
viewers if Network 2 shows a western. That is, this choice cannot guarantee more 
than 45 million viewers. Finally, if Network 1 shows a comedy, it would get 70 
million viewers if Network 2 is showing a comedy as well or as few as 14 million 
viewers if Network 2 is showing a soap opera. That is, this option cannot guarantee 
more than 14 million viewers. Clearly, the best option for Network 1 would be to 
show a soap opera, as at least 45 million viewers would tune into Network 1 
irrespective of what Network 2 does. 

In other words, the strategy for Network 1 (being a row player) is to determine 
the row minimum of each row and then to choose the row with the largest row 
minimum. 

Similarly, the best strategy for Network 2 (being a column player) is to deter- 
mine the column maximum of each column and then to choose the column with the 
smallest column maximum. For Network 2, the best option is to show a western and 
so 45 million viewers will tune into Network | to watch the soap opera, and 55 
million will tune into Network 2 to watch a western. 

It is clear from the table that the two outcomes satisfy the following inequality: 


Max (rows) (tow minimum) < Min(¢ojs) (col maximum). 


This choice is simultaneously best for Network 1 and Network 2 (as max(row 
minimum) = min(col maximum)), and this is called a saddle point, and the com- 
mon value of both sides of the equation is called the value of the game. An 
equilibrium point of the game is where there is a choice of strategies for both 
players where neither player can improve their outcome by changing their strategy, 
and a saddle point of a game is an example of an equilibrium point. 


Example (Prisoner Dilemma) 


The police arrest two people who they know have committed an armed robbery 
together. However, they lack sufficient evidence for a conviction for armed robbery, 
but they have sufficient evidence for a conviction of two years for the theft of the 
getaway car. The police make the following offer to each prisoner: 


If you confess to your part in the robbery and implicate your partner and he does not 
confess, then you will go free and he will get ten years. If you both confess you will both get 
five years. If neither of you confess you will get each get two years for the theft of the car 


Model the prisoners’ situation as a game and determine the rational (best pos- 
sible) outcome for each prisoner. 
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Solution (Prisoner’s Dilemma) 


There are four possible outcomes for each prisoner as follows: 


— Go Free (He confesses, Other does not). 

— 2-year sentence (Neither confess). 

5-year sentence (Both confess). 

10 years (He does not confess. Other does). 


Each prisoner has a choice of confessing or not, and Table 24.6 summarizes the 
various outcomes depending on the choices that the prisoners make. The first entry 
in each cell of the table is the outcome for prisoner 1 and the second entry is the 
outcome for prisoner 2. For example, the cell with entries 10, 0 states that prisoner 1 
is sentenced for 10 years and prisoner 2 goes free. 

It is clear from Table 24.6 that if both prisoners confess they both will receive, a 
5-year sentence; if neither confess, then they will both receive a 2-year sentence; if 
prisoner | confesses and prisoner 2 does not, then prisoner 1 goes free whereas 
prisoner 2 gets a 10-year sentence and finally, if prisoner 2 confesses and prisoner 1 
does not, then prisoner 2 goes free and prisoner a | gets a 10-year sentence. 

Each prisoner evaluates his two possible actions by looking at the outcomes in 
both columns, as this will show which action is better for each possible action of 
their partner. If prisoner 2 confesses, then prisoner 1 gets a 5-year sentence if he 
confesses or a 10-year sentence if he does not confess. If prisoner 2 does not 
confess, then prisoner | goes free if he confesses or 2 years if he does not confess. 
Therefore, prisoner | is better off confessing irrespective of the choice of prisoner 2. 
Similarly, prisoner 2 comes to exactly the same conclusion as prisoner |, and so the 
best outcome for both prisoners is to confess to the crime, and both will go to prison 
for 5 years. 

The paradox in the prisoners’ dilemma is that two individuals acting in their own 
self-interest do not produce the optimal outcome. Both parties choose to protect 
themselves at the expense of the other, and as a result both find themselves in a 
worse state than if they had cooperated with each other in the decision-making 
process and received two years. 

For more detailed information on operations research, see [2]. 


Table 24.6 Outcomes in 


: . Prisoner 2 
prisoners’ dilemma : 
Prisoner 1 Confess Refuse confess 
Confess 5}, 5) 0, 10 


Refuse confess 10, 0 2, 2 
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24.5 Review Questions 


e 


What is operations research? 

2. Describe the models used in operations research. 

3. What is linear programming and describe the steps in developing a linear 
programming model? 

4. What is cost-volume-profit analysis? 

5. Suppose the fixed costs are rent of £1,500 per month and that the cost of 
making each item is £20 and it may then be sold for £25. How many items 
must be sold to breakeven and what is the breakeven revenue? 

6. What is game theory? 

7. What is a zero sum game? 


24.6 Summary 


Operations research is a multi-disciplinary field concerned with the application of 
mathematical and analytic techniques to assist in decision-making. It employs 
mathematical modelling, statistical analysis and mathematical optimization to 
achieve optimal (or near-optimal) solutions to complex decision-making problems. 
The modern field of operations research includes other disciplines such as computer 
science, industrial engineering, business practices in manufacturing and service 
companies, supply chain management and operations management. 

Linear programming is a mathematical model for determining the best possible 
outcome such as maximizing profit or minimizing the cost of a particular problem. 
The problem is subject to various constraints such as resources or costs, and the 
constraints are expressed as a set of linear equations and linear inequalities. The 
best possible outcome is expressed as a linear equation. For example, the goal may 
be to determine the number of products that should be made to maximize profit 
subject to the constraint of limited available resources. 

Cost—volume-profit analysis (CVPA) is used in the analysis of the relationship 
between the costs, volume, revenue and profitability of the products produced. The 
relationship between revenue and costs at different levels of output can be displayed 
graphically, with revenue behaviour and cost behaviour shown graphically. 

Game theory is the study of mathematical models of strategic interaction among 
rational decision makers. Von Neumann was the founder of modern game theory 
with his 1928 paper on the theory of games of strategy. The Rand Corporation 
applied game theory to global nuclear strategy in the 1950s, and the original 
applications of game theory were to zero sum games where the gains or losses of 
each participant are exactly balanced by those of the other participants. 
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25.1 Introduction 


Interest is the additional payment that a borrower makes to the lender and is 
separate from the repayment of the principal. We distinguish between simple and 
compound interest, where simple interest is calculated on the principal only, 
whereas compound interest is calculated on both the principal and the accumulated 
interest of previous compounding periods. That is, simple interest is always cal- 
culated on the original principal, whereas for compound interest, the interest is 
added to the principal sum, so that interest is also earned on the added interest for 
the next compounding period. 

The future value is what the principal will amount to in the future at a given rate 
of interest, whereas the present value of an amount to be received in the future is the 
principal that would grow to that amount at a given rate of interest. 

An annuity is a sequence of fixed equal payments made over a period of time, 
and it is usually paid at the end of the payment interval. For example, for a hire 
purchase contract, the buyer makes an initial deposit for an item, and then pays an 
equal amount per month (the payment is generally at the end of the month) up to a 
fixed end date. Personal loans from banks are paid back in a similar manner but 
without an initial deposit. 

An interest-bearing debt is amortized if both the principal and interest are repaid 
by a series of equal payments (with the exception of possibly the last payment) 
made at equal intervals of time. The debt is repaid by an amortization annuity, 
where each payment consists of both the repayment of the capital borrowed and the 
interest due on the capital for that time interval. 
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25.2 Simple Interest 


Savers receive interest for placing deposits at the bank for a period of time, whereas 
lenders pay interest on their loans to the bank. Simple interest is generally paid on 
term deposits (these are usually short-term fixed-term deposits for 3, 6 or 12 months) 
or short-term investments or loans. The interest earned on a savings account depends 
on the principal amount placed on the deposit at the bank, the period of time that it will 
remain on deposit and the specified rate of interest for the period. 

For example, if Euro 5000 is placed on deposit at a bank with an interest rate of 
5% per annum for 2 years, then it will earn a total of Euro 500 in simple interest for 
the period. The interest amount is calculated by 


5000 * 5 * 2 


100 = Euro 500. 


The general formula for calculating the amount of simple interest A due for 
placing principal P on deposit at a rate r of interest (where r is expressed as a 
percentage) for a period of time T (in years) is given by 


_Pxrxt 
~ 100 - 


I 


If the rate of interest 7 is expressed as a decimal, then the formula for the interest 
earned is simply 


I=Pxrxt. 


It is essential in using the interest rate formula that the units for time and rate of 
interest are the same. 


Example (Simple Interest) 


Calculate the simple interest payable for the following short-term investments. 


1. £5000 placed on deposit for 6 months (EF year) at an interest rate of 4%. 
2. £3000 placed on deposit for 1 month Che year) at an interest rate of 5%. 
3. £10,000 placed on deposit for 1 day ('/365 year) at an interest rate of 7%. 


Solution (Simple Interest) 


1. A = 5000 * 0.04 * 0.5 = £100. 
2. A = 3000 * 0.05 * 0.08333 = £12.50. 
3. A= 10,000 * 0.07 * 0.00274 = £1.92. 


We may derive various formulae from the simple interest formula 
Il=PxrxT. 
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rt Pt Pr- 


Example (Finding the principal, rate or time) 

Find the value of the principal or rate or time in the following. 

1. What principal will earn interest of €24.00 at 4.00% in 8 months? 

2. Find the annual rate of interest for a principal of €800 to earn €50 in interest in 
9 months. 


3. Determine the number of months required for a principal of €2000 to earn €22 in 
interest at a rate of 5%. 


Solution 
We use the formulae derived from the simple interest formula to determine these. 


I 24 
rt 0.04 x 0.6666 
2. r= E = gps = 0.0833 = 8.33%. 


3. t= f= yew = 0.22 years = 2.64 months. 


1 P= = €900. 


25.2.1 Computing Future and Present Values 


The future value is what the principal will amount to in the future at a given rate of 
interest, whereas the present value of an amount to be received in the future is the 
principal that would grow to that amount at a given rate of interest. 


25.2.2 Computing Future Value 


A fixed-term account is an account that is opened for a fixed period of time 
(typically 3, 6 or 12 months). The interest rate is fixed during the term and thus, the 
interest due at the maturity date is known in advance. That is, the customer knows 
what the future value (FV) of the investment is and knows what is due on the 
maturity date of the account (this is termed the maturity value). 

On the maturity date, both the interest due and the principal are paid to the 
customer, and the account is generally closed on this date. In some cases, the 
customer may agree to roll over the principal, or the principal and interest for a 
further fixed period of time, but there is no obligation on the customer to do so. The 


426 25 Basic Financial Mathematics 


account is said to mature on the maturity date, and the maturity value (MV) or future 
value (FV) is given by the sum of the principal and interest: 


MV =FV=P4+HI. 
Further, since J = Prt, we can write this as MV = P + Prt or 


FV = MV = P(1+rt). 


Example (Computing Maturity Value) 


Jo invests €10,000 in a short-term investment for 3 months at an interest rate of 9%. 
What is the maturity value of her investment? 


Solution (Computing Maturity Value) 
MV = P(1+rt) 


= 10000(1 +0.09 x 0.25) 
= €10, 225. 


25.2.3 Computing Present Values 


The present value of an amount to be received at a given date in the future is the 
principal that will grow to that amount at a given rate of interest over that period of 
time. We computed the maturity value of a given principal at a given rate of interest 
r over a period of time ¢ as 


MV = P(1+rt). 


Therefore, the present value (PV = P) of an amount V to be received ¢ years in 
the future at an interest rate r (simple interest) is given by 


PV=P= 


(1+rt)” 


Example (Computing Present Value) 


Compute the present value of an investment eight months prior to the maturity date, 
where the investment earns interest of 7% per annum and has a maturity value of 
€920. 


Solution (Computing Present Value) 


V = 920, r = 0.07, t = 8/12 = 0.66. 
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V 920 
PV=P= = = € 879.37. 
(l+rt) (1+0.07*0.66) 


Example (Equivalent Values) 


Compute the payment due for the following: 


1. A payment of $2000 is due in 1 year. It has been decided to repay early and 
payment is to be made today. What is the equivalent payment that should be 
made given that the interest rate is 10%? 

2. A payment of $2000 is due today. It has been agreed that payment will instead 
be made 6 months later. What is the equivalent payment that will be made at 
that time given that the interest rate is 10%? 


Solution (Equivalent Values) 


1. The original payment date is 12 months from today and is now being made 
12 months earlier than the original date. Therefore, we compute the present 
value of $2000 for 12 months at an interest rate of 10%. 


V 2000 
PV=P= = = € 1818.18. 
(l+rt) (1+0.1*1) 


2. The original payment date is today but has been changed to 6 months later, and 
so we compute the future value of $2000 for 6 months at an interest rate of 10%. 


FV = P(1+/rt) = 2000(1+0.1 * 0.5) = 2000(1.05) = $2100. 


Example (Equivalent Values) 


A payment of €5000 that is due today is to be replaced by two equal payments (we 
call the unknown payments value x) due in 4 and 8 months, where the interest rate 
is 10%. Find the value of the replacement payments. 


Solution (Equivalent Values) 


The sum of the present value of the two (equal but unknown) payments is €5000. 
The present value of x (received in 4 months) is 


4 x x 


PV=P= = = = 0.9678x. 
(+n) (1+0.1 * 0.33) 1.033 7 
The present value of x (received in 8 months) is 
4 x x 
(i+rt) (1+0.1 * 0.66) 1.066 7 
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Therefore, 


0.9678x + 0.9375x = €5000 
=> 1.9053x = €000 
=> x = €2624.26. 


25.3 Compound Interest 


The calculation of compound interest is more complicated as may be seen from the 
following example: 


Example (Compound Interest) 


Calculate the interest earned and what the new principal will be on Euro 1000, 
which is placed on deposit at a bank, with an interest rate of 10% per annum 
(compound) for 3 years. 


Solution 


At the end of year 1, Euro 100 of interest is earned, and this is added to the existing 
principal making the new principal (at the start of year 2) €1000 + €100 = Euro 
1100. At the end of year 2, Euro 110 is earned in interest, and this is added to the 
principal making the new principal (at the start of year 3) €1100 + €110 = Euro 
1210. Finally, at the end of year 3, a further Euro 121 is earned in interest, and so 
the new principal is Euro 1331, and the total interest earned for the 3 years is the 
sum of the interest earned for each year (i.e. Euro 331). This may be seen in 
Table 25.1 below. 

The new principal each year is given by a geometric sequence (recall a geometric 
sequence is a sequence in the form a, ar, ar’, ... ar"). For this example, we have 
a = 1000, and as the interest rate is 10% = Ve = 0.1, we have r = (1 + 0.1), and 
so the sequence is 


1000, 1000(1.1), 1000(1.1)?, 1000(1.1)°, .... 


Table 25.1. Calculation of 


Aa Year Principal Interest earned New principal 
a I €1000 €100 €1100 
2 €1100 €110 €1210 


3 €1210 €121 €1331 
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That is, if a principal amount P is invested for n years at a rate r of interest (r is 
expressed as a decimal), then it will amount to 


A=FV=P(1+7r)". 
For our example above, A = 1000, t = 3 and r = 0.1 Therefore, 


A = 1000(1.1)° 
= €1331(as before). 


A principal amount P invested for n years at a rate r of simple interest (r is 
expressed as a decimal) will amount to 


A=FV=P(1+7rt). 


The principal €1000 invested for 3 years at a rate of interest of 10% (simple 
interest) will amount to 


A = 1000(1+0.1 * 3) = 1000(1.3) = €1300. 


There are variants of the compound interest formula to cover situations where 
there are m-compounding periods per year. For example, interest may be com- 
pounded annually, semi-annually (with 2 compounding periods per year), quarterly 
(with 4 compounding periods per year), monthly (with 12 compounding periods per 
year) or daily (with 365 compounding periods per year). 

The periodic rate of interest (i) per compound period is given by the nominal 
annual rate of interest (7) divided by the number of compounding periods (m) per 
year: 


Nominal Rate r 


= # compounding periods im 


For example, if the nominal annual rate is 10% and interest is compounded 
quarterly, then the period rate of interest per quarter is '°/, = 2.5%. That is, com- 
pound interest of 2.5% is calculated at the end of each quarter and applied to the 
account. 

The number of compounding periods for the total term of a loan or investment is 
given by the number of compounding periods per year (m) multiplied by the 
number of years of the investment or loan. 


n= years X m. 
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Example (Compound Interest—Multiple Compounding Periods) 


An investor places £10,000 on a term deposit that earns interest at 8% per annum 
compounded quarterly for 3 years and 9 months. At the end of the term, the interest 
rate changes to 6% compounded monthly and it is invested for a further term of 
2 years and 3 months. 


BWNe 


. How many compounding periods are there for 3 years and 9 months? 

. What is the value of the investment at the end of 3 years and 9 months? 
. How many compounding periods are there for 2 years and 3 months? 

. What is the final value of the investment at the end of the 6 years? 


Solution (Compound Interest—Multiple Compounding Periods) 


1. 


The initial term is for 3 years and 9 months (i.e. 3.75 years), and so the total 
number of compounding periods is given by n = #years * m, where #years = 
3.75 and m = 4. Therefore, n = 3.75 * 4 = 15. 


. The nominal rate of interest r is 8% = 0.08, and so the interest rate i per 


quarterly compounding period is 0.08/4 = 0.02. 
Therefore, at the end of the term, the principal amounts to 


A= FV, =P(1+i)" 
= 10000(1 + 0.02)'° 


= 10000(1.02)'° 
= 10000(1.3458) 
= £13,458. 


. The term is for 2 years and 3 months (i.e. 2.25 years), and so the total number 


of compounding periods is given by n = #years * m, where #years = 2.25 and 
m = 12. Therefore, n = 2.25 * 12 = 27. 


. The new nominal interest rate is 6% = 0.06 and so the interest rate i per 


compounding period is °°°/;, = 0.005. Therefore, at the end of the term, the 
principal amounts to 
A= FV, = FV,(1 +i)" 
= 13458(1 +0.005)”” 


= 13458(1.005)”” 
= 13458 * 1.14415 
= £15,398. 
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25.3.1 Present Value Under Compound Interest 


The time value of money is the concept that the earlier that cash is received, the 
greater its value to the recipient. Similarly, the later that a cash payment is made, the 
lower its value to the payee and the lower its cost to the payer. 

This is clear if we consider the example of a person who receives $1000 now and 
a person who receives $1000 five years from now. The person who receives $1000 
now is able to invest it and to receive compound interest on the principal, whereas 
the other person who receives $1000 in five years earns no interest during the 
period. Further, the inflation during the period means that the purchasing power of 
$1000 is less in five years time than it is today. 

We presented the general formula for the future value of a principal P invested 
for n compounding periods at a compound rate r of interest per compounding 
period as 


A=P(1+r)". 


The present value of a given amount A that will be received in the future is the 
principal (P = PV) that will grow to that amount where there are n compounding 
periods and the rate of interest is r for each compounding period. The present value 
of an amount A received in n compounding periods at an interest rate r for the 
compounding period is given by 

A 


P= 


We can also write the present value formula as PV = P = A(1 +r)”. 
Example (Present Value) 


Find the principal that will amount to $10,000 in 5 years at 8% per annum com- 
pounded quarterly. 


Solution (Present Value) 


The term is 5 years = 5 * 4 = 20 compounding period. The nominal rate of interest 
is 8% = 0.08 and so the interest rate i per compounding period is 0.087, = 0.02. The 
present value is then given by 


PV =A(1+i) "= FV(1+i)” 
= 10000(1.02)~*° 
= $6729.71. 


The difference between the known future value of $10,000 and the computed 
present value (i.e. the principal of $6729.71) is termed the compound discount and 
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represents the compound interest that accumulates on the principal over the period 
of time. It is given by 


Compound Discount = FV — PV. 


For this example, the compound discount is $10,000 — 6729.71 = $3270.29. 
Example (Present Value) 


Elodie is planning to buy a home entertainment system for her apartment. She can 
pay £1500 now or pay £250 now and £1600 in 2 years time. Which option is better 
if the nominal rate of interest is 9% compounded monthly? 


Solution (Present Value) 


There are 2 * 12 = 24 compounding periods and the interest rate i for the com- 
pounding period is °°? /,, = 0.0075. The present value of £1600 in two years time 
at an interest rate of 9% compounded monthly is 


PV = FV(1+i)" 
= 1600(1.0075)** 
= 1600/1.1984 
= £1337.33. 


The total cost of the second option is £250 + 1337.33 = £1587.33. 
Therefore, Elodie should choose the first option since it is cheaper by £87.33 (i.e. 
£1587.33 — 1500). 


25.3.2 Equivalent Values 


When two sums of money are to be paid/received at different times, they are not 
directly comparable as such, and a point in time (the focal date) must be chosen to 
make the comparison (Fig. 25.1). 

The choice of focal date determines whether the present value or future value 
formula will be used. That is, when computing equivalent values, we first determine 
the focal date, and then depending on whether the payment date is before or after 
this reference date, we apply the present value or future value formula. 

If the due date of the payment is before the focal date, then we apply the future 
value FV formula: n 


FV = P(1+i)". 
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Fig. 25.1 Equivalent 
weights 


If the due date of the payment is after the focal date, then we apply the present 
value PV formula: 


PV = FV(1+i)”. 


Example (Equivalent Values) 


A debt value of €1000 that was due 3 months ago, €2000 that is due today and 
€1500 that is due in 18 months are to be combined into one payment due 6 months 
from today at 8% compounded monthly. Determine the amount of the single 
payment. 


Solution (Equivalent Values) 

The focal date is 6 months from today and so we need to determine the 
equivalent value E;, Ez and E; of the three payments on this date, and we then 
replace the three payments with one single payment E = E; + E, + E; that is 
payable 6 months from today. 

The equivalent value E, of €1000 which was due 3 months ago in 6 months from 
today is determined from the future value formula where the number of interest 
periods n = 6 + 3 = 9. The interest rate per period is 8%/12 = 0.66% = 0.00667. 


FV = P(1+i)" 
= 1000(1 + 0.00667)” 


= 1000(1.00667)° 
= 1000 * 1.0616 
= €1061.60. 
The equivalent value E, of €2000 which is due today in 6 months is determined 


from the future value formula, where the number of interest periods n = 6. The 
interest rate per period is = 0.00667. 
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FV = P(1+i)" 
= 2000(1 + 0.00667)° 
= 2000(1.00667)° 
= 2000 * 1.0407 
= €2081.40. 


The equivalent value E3 of €1500 which is due in 18 months from today is 
determined from the present value formula, where the number of interest periods 
n= 18 — 6 = 12. The interest rate per period is = 0.00667. 


PV =FV(1+i)” 
= 1500(1 + 0.0067) ~"* 


= 1500(1.0067) 
= €1384.49, 


E=E,+E2+£, 
= 1061.60 + 2081.40 + 1384.49 
= £4527.49. 


And so, a single equivalent payment of €4527.49 is paid in 6 months. 
Example (Equivalent Values—Replacement Payments) 


Liz was due to make a payment of £2000 today. However, she has negotiated a deal 
to make two equal payments: the first payment is to be made one year from now 
and the second payment two years from now. Determine the amount of the equal 
payments where the interest rate is 9% compounded quarterly and the focal date is 
today. 


Solution (Equivalent Values—Replacement Payments) 


Let x be the value of the equal payments. The first payment is made in one year and 
so there are n = 1 * 4 = 4 compounding periods and the second payment is made 
in 2 years and so there are n = 2 * 4 = 8 compounding periods. The interest rate 
iis 9%/4 = 2.25% = 0.0225. 

The present value E, of a sum x received in one year is given by 


PV = FV(1+i)” 
= x(1+0,0225)~* 


= x(1.0225)~* 
= 0.9148x. 
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The present value E, of a sum x received in two years is given by 


PV = FV(1+i)” 
= x(1+0.0225) 7 
= x(1.0225) 8 
= 0.8369x. 


The sum of the present value of E, and E, is £2000 and so we have 


0.9148x + 0.8369x = 2000 
1.7517x = 2000 
x = £1141.75. 


25.4 Basic Mathematics of Annuities 


An annuity is a sequence of fixed equal payments made over a period of time, and it 
is usually paid at the end of the payment interval. For example, for a hire purchase 
contract, the buyer makes an initial deposit for an item and then pays an equal 
amount per month (the payment is generally at the end of the month) up to a fixed 
end date. Personal loans from banks are paid back in a similar manner but without 
an initial deposit. 

An investment annuity (e.g. a regular monthly savings scheme) may be paid at 
the start of the payment interval. A pension scheme involves two stages, with the 
first stage involving an investment of regular payments at the start of the payment 
interval up to retirement and the second stage involving the payment of the 
retirement annuity. The period of payment of a retirement annuity is usually for the 
remainder of a person’s life (life annuity) or it could be for a period of a fixed 
number of years. 

We could determine the final value of an investment annuity by determining the 
future value of each payment up to the maturity date and then adding them all 
together. Alternately, as the future values form a geometric series, we may derive a 
formula for the value of the investment by using the formula for the sum of a 
geometric series. 

We may determine the present value of an annuity by determining the present 
value of each payment made and summing these or we may also develop a formula 
to calculate the present value. 

The repayment of a bank loan is generally with an amortization annuity, where 
the customer borrows a sum of money from a bank (e.g. a personal loan for a car or 
a mortgage for the purchase of a house). The loan is for a defined period of time and 
its repayment is with a regular annuity, where each annuity payment consists of 


436 25. Basic Financial Mathematics 


interest and capital repayment. The bulk of the early payments goes on the interest 
due on the outstanding capital with smaller amounts going on capital repayments. 
However, the bulk of the later payments goes in repaying the capital with smaller 
amounts going on interest. 


Definition 


An annuity is a series of equal cash payments made at regular intervals over a 
period of time, and they may be used for investment purposes or paying back a loan 
or mortgage. We first consider the example of an investment annuity. 


Example (Investment Annuity) 


Sheila is investing €10,000 a year in a savings scheme that pays 10% interest every 
year. What will the value of her investment be after 5 years? 


Solution (Invested Annuity) 


Sheila invests €10,000 at the start of year 1 and so this earns 5 years of compound 
interest of 10% and so its future value in 5 years is given by 10,000 * 11° = 
€16,105. The future value of the payments that she makes is presented in 
Table 25.2. 

Therefore, the value of her investment at the end of 5 years is the sum of the 
future values of each payment at the end of 5 years = 16,105 + 14,641 + 
13,310 + 12,100 + 11,000 = €67,156. 

We note that this is the sum of a geometric series and so in general if an investor 
makes a payment of A at the start of each year for n years at a rate r of interest, then 
the investment value at the end of n years is 


A(L+r)"+A(1 +r)" !4+---A(L +7) 


=A(1+r)[1 4A +0) 4 +AU +0)" 
(ayy a 
= A(1 +r) ———_ 
( ) (l+r)-1 
l+n"-1 
Ayer at 
r 
Table 25.2 Calculation of Year Amount Future value (r = 0.1) 
future value of annuity 5 
1 10,000 10,000 * 1.1° = €16,105 
2 10,000 10,000 * 1.1* = €14,641 
3 10,000 10,000 * 1.13 = €13,310 
4 10,000 10,000 * 1.17 = €12,100 
5 10,000 10,000 * 1.1' = €11,000 


Total €67,156 
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We apply the formula to check our calculation. 


(fot 1 


10000(1 +0.1) = 


(4 
0.1 
(1.61051 — 1) 
= 11000 + 
0.1 


= €67, 156. 


Note 25.1 


We assumed that the annual payment was made at the start of the year. However, 
for ordinary annuities, payment is made at the end of the year (or compounding 
period) and so the formula would be slightly different: 
1 "_ 4] 
FV=—A (+r)! -1 
ie 

The future value formula is adjusted for multiple (7) compounding periods per 
year, where the interest rate for the period is given by i = ’/,,, and the number of 
payment periods n is given by where n = tm (where ¢ is the number of years). The 
future value of a series of payments of amount A (made at the start of the com- 


pounding period) with interest rate i per compounding period, where there are 
n compounding periods, is given by 


+)" -1 


FV = A(1+i) ; 

The future value of a series of payments of amount A (made at the end of the 
compounding period) with interest rate 7 per compounding period, where there are 
n compounding periods, is given by 


14+7)"-1 
page) 
Ll 


An annuity consists of a series of payments over a period of time, and so it is 
reasonable to consider its present value with respect to a discount rate r (this is 
applicable to calculating the present value of the annuity for mortgage repayments 
discussed in the next section). 

The net present value of an annuity is the sum of the present value of each of the 
payments made over the period, and the method of calculation is clear from 
Table 25.3 below. 
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Table 25.3 Calculation of 


Year Amount Present value (r = 0.1) 
present value of annuity 1 1000 $909.91 

2 1000 $826.44 

3 1000 $751.31 

4 1000 $683.01 

5) 1000 $620.92 

Total $3791 


Example (Present Value Annuities) 


Calculate the present value of a series of payments of $1000 with the payments 
made for 5 years at a discount rate of 10%. 


Solution (Present Value Annuities) 


The regular payment A is 1000, the rate r is 0.1 and n = 5. The present value of the 
first payment received is 1000/1.1 = 909.91 at the end of the year 1; at the end of 
year 2, it is 1000/(1.1)* = 826.45 and so on. At the end of year 5, its present value is 
620.92. The net present value of the annuity is the sum of the present value of all 
the payments made over the five years, and it is given by the sum of the present 
values from Table 25.3. That is, the present value of the annuity is 
909.91 + 826.44 + 751.31 + 683.01 + 620.92 = $3791. 

We may derive a formula for the present value of a series of payments A made 
over a period of n years at a discount rate of r as follows: Clearly, the present value 
is given by 


A ’ A A 
(1+r) 


T (+7 eee +n” 


1 


This is a geometric series where the constant ratio is ee 


and the present value of 
the annuity is given by its sum: 


PV = a 1- eared! ; 
r (1+r) 
For the example above, we apply the formula and get 
1000 1 
PV = ——_ |1- : 
0.1 (1.1) 


= 10000(0.3791) 
= $3791. 
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The annuity formula is adjusted for multiple (7) compounding periods per year, 
and the interest rate for the period is given by i = ’/,,, and the number of payment 
periods n is given by n = tm (where t¢ is the number of years). For example, the 
present value of an annuity of amount A, with interest rate i per compounding 
period, where there are n compounding periods, is given by 


rotate 


Example (Retirement Annuity) 


Blaithin has commenced employment at a company that offers a pension in the form of 
an annuity that pays 5% interest per annum compounded monthly. She plans to work for 
30 years and wishes to accumulate a pension fund that will pay her €2000 per month for 
25 years after she retires. How much does she need to save per month to do this? 


Solution (Retirement Annuities) 


First, we determine the value that the fund must accumulate to pay her €2000 per 
month, and this is given by the present value of the 25 years annuity of €2000 per 
month. The interest rate r is 5% and as there are 12 compounding periods per year, 
there are a total of 25 * 12 = 300 compounding periods, and the interest rate per 
compounding period is 0.05/12 = 0.004166. 


P= 2000/0.004166} | — (1.004166) *°° 
= €342, 174. 


That is, her pension fund at retirement must reach €342,174 and so we need to 
determine the monthly payments necessary for her to achieve this. The future value 
is given by the formula: 


(14+ eee —] 


I 


FV=A 


and so 


A = Fvi/[(+iy""’ =e 


where m = 12, n = 30 * 12 = 360 and i = 0.05/12 = 0.004166 and FV = 342,174. 
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A = 342,174 « 0.004166/3.4863 
= €408.87. 


That is, Blaithin needs to save €408.87 per month into her retirement account 
(sinking fund) for 30 years in order to have an annuity of €2000 per month for 
25 years (where there is a constant interest rate of 5% compounded monthly). 


25.5 Loans and Mortgages 


The purchase of a home or car requires a large sum of money, and so most 
purchasers need to obtain a loan from the bank to fund the purchase. Once the 
financial institution has approved the loan, the borrower completes the purchase and 
pays back the loan to the financial institution over an agreed period of time (the 
term of the loan). For example, a mortgage is generally paid back over 20-25 years, 
whereas a car loan is usually paid back in 5 years. (Fig. 25.2) 

An interest-bearing debt is amortized if both the principal and interest are repaid 
by a series of equal payments (with the exception of possibly the last payment) 
made at equal intervals of time. That is, the amortization of loans refers to the 
repayment of interest-bearing debts by a series of equal payments made at equal 
intervals of time. The debt is repaid by an amortization annuity, where each pay- 
ment consists of both the repayment of the capital borrowed and the interest due on 
the capital for that time interval. 

Mortgages and many consumer loans are repaid by this method, and the standard 
problem is to calculate what the annual (or monthly) payment should be to amortize 
the principal P in n years where the rate of interest is r. 

The present value of the annuity is equal to the principal borrowed, i.e. the sum 
of the present values of the payments must be equal to the original principal 
borrowed. That is, 


Fig. 25.2 Loan or mortgage 
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We may also use the formula that we previously derived for the present value of 


the annuity to get 
A 1 
P =——_ 1 Fan | + 
r (1+r) 


We may calculate A by manipulating this formula to get 
Pr 
1 
[! — oh 


A= Pr/{l—(1+r)”"]. 


A= 


Example (Amortization) 


Joanne has taken out a €200,000 mortgage over 20 years at 8% per annum. Cal- 
culate her annual repayment amount to amortize the mortgage. 


Solution (Amortization) 


We apply the formula to calculate her annual repayment: 


_ 200000 * 0.08 


[\— 
(1 +0.08)”° 
_ 16000 
[1 — eer] 
= €20, 370. 
We adjust the formula for the more common case where the interest is com- 


pounded several times per year (usually monthly), and so n = #years * #com- 
poundings and the interest i = r/#compoundings. 
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Example (Amortization) 


A mortgage of £150,000 at 6% compounded monthly is amortized over 20 years. 
Determine the following: 


1. Repayment amount per month. 
2. Total amount paid to amortize the loan. 
3. The cost of financing. 


Solution (Amortization) 


The number of payments n = #years * payments per year = 20 * 12 = 240. 
The interest rate i = 6%/12 = 0.5% = 0.005. 


1. We calculate the amount of the repayment A by substituting for m and i and obtain 


j= 150000 * 0.005 
Grats 
(1+0.005)*” 

750 


[1 — 33100] 


= £1074.65. 


2. The total amount paid is the number of payments * amount of each payment = 
n* A = 240 * 1074.65 = £257,916. 

3. The total cost of financing = total amount paid — original principal = 
257,916 — 150,000 = £107,916. 


Example (Amortization) 
For the previous example, determine the following at the end of the first period: 


1. The amount of interest repaid. 
2. The amount of the principal repaid. 
3. The amount of the principal outstanding. 


Solution (Amortization) 


The amount paid at the end of the first period is £1074.65. 


1. The amount paid in interest for the first period is 150000 * 0.005 = £750. 

2. The amount of the principal repaid is £1074 — 750 = £324.65. 

3. The amount of the principal outstanding at the end of the first interest period is 
£150,000 — 324.65 = £149,675.35. 


25.5 Loans and Mortgages 443 


The early payments of the mortgage mainly involve repaying interest on the 
capital borrowed, whereas the later payments mainly involve repaying the capital 
borrowed with less interest due. We can create an amortization table that shows the 
interest paid, the amount paid in capital repayment and the outstanding principal 
balance for each payment interval. Often, the last payment is different from the 
others due to rounding errors introduced and carried through. 

Each entry in the amortization table includes interest, principal repaid and out- 
standing principal balance. The interest is calculated by the principle bal- 
ance * periodic interest rate i; the principal repaid is calculated by the payment 
amount—interest; the new outstanding principal balance is given by the principal 
balance—principal repaid. 


25.6 Review Questions 


— 


. Explain the difference between simple and compound interest? 

2. Calculate the simple interest payable on an investment of £12,000 placed 
on deposit for 9 months at an interest rate of 8%. 

3. An investor places £5,000 on a term deposit that earns interest at 8% per 
annum compounded quarterly for 2 years and 3 months. Find the value of 
the investment on the maturity date. 

4. Find the principal that will amount to $12,000 in 3 years at 6% per annum 
compounded quarterly. 

5. How many years will it take a principal of $5000 to exceed $10,000 at a 
constant annual growth rate of 6% compound interest? 

6. What is the present value of $5000 to be received in 5 years time at a 
discount rate of 7%? 

7. Explain the concept of equivalent values and when to use the present 
value/future value in its calculation. 

8. A debt value of €2000 due 6 months ago, €5000 due today and €3000 due 

in 18 months are to be combined into one payment due 3 months from 

today at 6% compounded monthly. Determine the amount of the single 
payment. 
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25.7. Summary 


Simple interest is calculated on the principal only whereas compound interest is 
calculated on both the principal and the accumulated interest of previous periods. 
Compound interest is generally used for long-term investments and loans, and its 
calculation is more complicated than that of simple interest. 

The time value of money is the concept that the earlier that cash is received, the 
greater its value to the recipient, and vice versa for late payments. The future value 
of a principal P invested for n year compounding periods at a compound rate r of 
interest per compounding period is given by A = P (1 + r)”. The present value of a 
given amount A that will be received in n years in the future at an interest rate r for 
each compounding period is the principal that will grow to that amount and is given 
by P=AQ +n)”. 

The future value is what the principal will amount to in the future at a given rate 
of interest, whereas the present value of an amount to be received in the future is the 
principal that would grow to that amount at a given rate of interest. 

An annuity is a sequence of fixed equal payments made over a period of time, 
and it is usually paid at the end of the payment interval. An interest-bearing debt is 
amortized if both the principal and interest are repaid by a series of equal payments 
(with the exception of possibly the last payment) made at equal intervals of time. 
The debt is repaid by an amortization annuity, where each payment consists of both 
the repayment of the capital borrowed and the interest due on the capital for that 
time interval. 
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